{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Part 2: Flexibility Make the Life Much More Easier -- LSTM Playground"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LSTM is verty important for us. \n",
    "\n",
    "LSTM is our baseline model, we can compare different architecture with LSTM model to evaluate whether others model is worth to use.\n",
    "\n",
    "There are two interesting things we have done for LSTM model:\n",
    "\n",
    "### 1. We want to be faster\n",
    "A big difficulty we have to deal with is Even LSTM Model takes so long to train. We need to do great bunch of experiments, but if we use the pipline out there on kaggle kernel, whatever change we do will takes 2 hours to evaluate.\n",
    "\n",
    "We found that one optimization we can do to train the model faster is reduce the padding length. The current padding length for the sentences is 220. This is longer than 99% of the sentences in the dataset, but most of the sample length is not longer than 100.\n",
    "\n",
    "So, we sort the dataset by sentences length and group similar length sentences together. Therefore at training stage, we can do different sequence padding by the longest sentences in the group, which save us a lot of time.\n",
    "\n",
    "![padding](Pictures/padding.png)",
    "<br/>\n",
    "<br/>\n",
    "\n",
    "### 2. Statistical Features is worth to try\n",
    "Acutally,statistical Feature didn't give us any boost finally. \n",
    "\n",
    "However, it seems that adding statistical features to the model will have a higher score at first 3 epoch, when the model is converge, the finally score won't better than non-stats LSTM model. \n",
    "\n",
    "We still think it's interesting finding. Maybe stats feature is easy to capture and make prediction with by the model, but they are not strong features. Therefore in later training stage, this kinds of features was outperformed by better features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fast LSTM Model With Statistical Features\n",
    "\n",
    "The following model is a interesting pipline we build that can easily train a good LSTM in a relatively short time using the techique just mentioned. (Comparable LSTM in Kaggle Hot Kernel takes about **about 1 hours** to train. But this pipline only takes **20 mins** per model.)\n",
    "\n",
    "Model Architecture:\n",
    "![LSTM](Pictures/LSTM.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-02T23:36:13.748934Z",
     "start_time": "2019-06-02T23:36:07.877720Z"
    }
   },
   "outputs": [],
   "source": [
    "from contextlib import contextmanager\n",
    "import os\n",
    "import random\n",
    "import re\n",
    "import string\n",
    "import time\n",
    "import warnings\n",
    "from tqdm._tqdm_notebook import tqdm_notebook as tqdm\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "pd.set_option('display.max_rows', 500)\n",
    "pd.set_option('display.max_columns', 500)\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "from sklearn import metrics\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.utils.data\n",
    "from torch.optim.optimizer import Optimizer\n",
    "from torch.nn import functional as F\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-02T23:36:25.460303Z",
     "start_time": "2019-06-02T23:36:13.751895Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1804874"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pd.read_csv(TRAIN_DATA))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-02T23:35:40.392856Z",
     "start_time": "2019-06-02T23:35:40.082376Z"
    }
   },
   "outputs": [],
   "source": [
    "EMBEDDING_FASTTEXT = './input/crawl-300d-2M.vec'\n",
    "TRAIN_DATA = './data/train.csv'\n",
    "TEST_DATA = './data/test.csv'\n",
    "SAMPLE_SUBMISSION = './data/sample_submission.csv'\n",
    "\n",
    "embed_size = 300\n",
    "max_features = 1000000\n",
    "maxlen = 220\n",
    "\n",
    "batch_size = 512\n",
    "train_epochs = 8\n",
    "n_splits = 5\n",
    "\n",
    "seed = 2333"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Fuctions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:13.166079Z",
     "start_time": "2019-06-01T04:06:13.160136Z"
    }
   },
   "outputs": [],
   "source": [
    "def custom_loss(data, targets):\n",
    "    ''' Define custom loss function for weighted BCE on 'target' column '''\n",
    "    bce_loss_1 = nn.BCEWithLogitsLoss(weight=targets[:,1:2])(data[:,:1],targets[:,:1])\n",
    "    bce_loss_2 = nn.BCEWithLogitsLoss()(data[:,1:],targets[:,2:])\n",
    "    return (bce_loss_1 * loss_weight) + bce_loss_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:13.496757Z",
     "start_time": "2019-06-01T04:06:13.493789Z"
    }
   },
   "outputs": [],
   "source": [
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:13.825976Z",
     "start_time": "2019-06-01T04:06:13.821987Z"
    }
   },
   "outputs": [],
   "source": [
    "@contextmanager\n",
    "def timer(msg):\n",
    "    t0 = time.time()\n",
    "    print(f'[{msg}] start.')\n",
    "    yield\n",
    "    elapsed_time = time.time() - t0\n",
    "    print(f'[{msg}] done in {elapsed_time / 60:.2f} min.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:14.181027Z",
     "start_time": "2019-06-01T04:06:14.174047Z"
    }
   },
   "outputs": [],
   "source": [
    "def seed_torch(seed=1029):\n",
    "    random.seed(seed)\n",
    "    os.environ['PYTHONHASHSEED'] = str(seed)\n",
    "    np.random.seed(seed)\n",
    "    torch.manual_seed(seed)\n",
    "    torch.cuda.manual_seed(seed)\n",
    "    torch.backends.cudnn.deterministic = True\n",
    "seed_torch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:14.509183Z",
     "start_time": "2019-06-01T04:06:14.502172Z"
    }
   },
   "outputs": [],
   "source": [
    "puncts = [',', '.', '\"', ':', ')', '(', '-', '!', '?', '|', ';', \"'\", '$', '&', '/', '[', ']',\n",
    "          '>', '%', '=', '#', '*', '+', '\\\\', '•', '~', '@', '£', '·', '_', '{', '}', '©', '^',\n",
    "          '®', '`', '<', '→', '°', '€', '™', '›', '♥', '←', '×', '§', '″', '′', 'Â', '█',\n",
    "          '½', 'à', '…', '“', '★', '”', '–', '●', 'â', '►', '−', '¢', '²', '¬', '░', '¶',\n",
    "          '↑', '±', '¿', '▾', '═', '¦', '║', '―', '¥', '▓', '—', '‹', '─', '▒', '：', '¼',\n",
    "          '⊕', '▼', '▪', '†', '■', '’', '▀', '¨', '▄', '♫', '☆', 'é', '¯', '♦', '¤', '▲',\n",
    "          'è', '¸', '¾', 'Ã', '⋅', '‘', '∞', '∙', '）', '↓', '、', '│', '（', '»', '，', '♪',\n",
    "          '╩', '╚', '³', '・', '╦', '╣', '╔', '╗', '▬', '❤', 'ï', 'Ø', '¹', '≤', '‡', '√']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:14.859215Z",
     "start_time": "2019-06-01T04:06:14.852234Z"
    }
   },
   "outputs": [],
   "source": [
    "misspell_dict = {\"aren't\": \"are not\", \"can't\": \"cannot\", \"couldn't\": \"could not\",\n",
    "                 \"didn't\": \"did not\", \"doesn't\": \"does not\", \"don't\": \"do not\",\n",
    "                 \"hadn't\": \"had not\", \"hasn't\": \"has not\", \"haven't\": \"have not\",\n",
    "                 \"he'd\": \"he would\", \"he'll\": \"he will\", \"he's\": \"he is\",\n",
    "                 \"i'd\": \"I had\", \"i'll\": \"I will\", \"i'm\": \"I am\", \"isn't\": \"is not\",\n",
    "                 \"it's\": \"it is\", \"it'll\": \"it will\", \"i've\": \"I have\", \"let's\": \"let us\",\n",
    "                 \"mightn't\": \"might not\", \"mustn't\": \"must not\", \"shan't\": \"shall not\",\n",
    "                 \"she'd\": \"she would\", \"she'll\": \"she will\", \"she's\": \"she is\",\n",
    "                 \"shouldn't\": \"should not\", \"that's\": \"that is\", \"there's\": \"there is\",\n",
    "                 \"they'd\": \"they would\", \"they'll\": \"they will\", \"they're\": \"they are\",\n",
    "                 \"they've\": \"they have\", \"we'd\": \"we would\", \"we're\": \"we are\",\n",
    "                 \"weren't\": \"were not\", \"we've\": \"we have\", \"what'll\": \"what will\",\n",
    "                 \"what're\": \"what are\", \"what's\": \"what is\", \"what've\": \"what have\",\n",
    "                 \"where's\": \"where is\", \"who'd\": \"who would\", \"who'll\": \"who will\",\n",
    "                 \"who're\": \"who are\", \"who's\": \"who is\", \"who've\": \"who have\",\n",
    "                 \"won't\": \"will not\", \"wouldn't\": \"would not\", \"you'd\": \"you would\",\n",
    "                 \"you'll\": \"you will\", \"you're\": \"you are\", \"you've\": \"you have\",\n",
    "                 \"'re\": \" are\", \"wasn't\": \"was not\", \"we'll\": \" will\", \"tryin'\": \"trying\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:15.274996Z",
     "start_time": "2019-06-01T04:06:15.271025Z"
    }
   },
   "outputs": [],
   "source": [
    "# re.sub() 和第一个参数match的东西 会返回一个re相关的object 传入第二个参数的方程中（如果第二个参数是方程）\n",
    "\n",
    "def _get_misspell(misspell_dict):\n",
    "    misspell_re = re.compile('(%s)' % '|'.join(misspell_dict.keys()))\n",
    "    return misspell_dict, misspell_re\n",
    "\n",
    "\n",
    "def replace_typical_misspell(text):\n",
    "    misspellings, misspellings_re = _get_misspell(misspell_dict)\n",
    "\n",
    "    def replace(match):\n",
    "        return misspellings[match.group(0)]\n",
    "\n",
    "    return misspellings_re.sub(replace, text) ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:15.801037Z",
     "start_time": "2019-06-01T04:06:15.794054Z"
    }
   },
   "outputs": [],
   "source": [
    "def clean_text(x): ### 用空格把标点符号两边的内容隔开\n",
    "    x = str(x)\n",
    "    for punct in puncts + list(string.punctuation):\n",
    "        if punct in x:\n",
    "            x = x.replace(punct, f' {punct} ')\n",
    "    return x\n",
    "\n",
    "\n",
    "def clean_numbers(x): #### 数字直接删掉\n",
    "    return re.sub('\\d+', ' ', x)\n",
    "\n",
    "\n",
    "def get_coefs(word, *arr):\n",
    "    return word, np.asarray(arr, dtype='float32')\n",
    "\n",
    "\n",
    "def load_fasttext(word_index):\n",
    "    embeddings_index = dict(get_coefs(*o.strip().split(' ')) for o in open(EMBEDDING_FASTTEXT,encoding='utf8'))\n",
    "\n",
    "    # word_index = tokenizer.word_index\n",
    "    nb_words = min(max_features, len(word_index)+1)\n",
    "    embedding_matrix = np.zeros((nb_words, embed_size))\n",
    "\n",
    "    for word, i in word_index.items():\n",
    "        if i >= max_features:\n",
    "            continue\n",
    "        embedding_vector = embeddings_index.get(word)\n",
    "        if embedding_vector is not None:\n",
    "            embedding_matrix[i] = embedding_vector\n",
    "\n",
    "    return embedding_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:16.231886Z",
     "start_time": "2019-06-01T04:06:16.228861Z"
    }
   },
   "outputs": [],
   "source": [
    "# Convert identity columns to booleans\n",
    "def convert_to_bool(df, col_name):\n",
    "    df[col_name] = np.where(df[col_name] >= 0.5, True, False)\n",
    "    \n",
    "def convert_dataframe_to_bool(df):\n",
    "    bool_df = df.copy()\n",
    "    for col in identity_columns:\n",
    "        convert_to_bool(bool_df, col)\n",
    "    return bool_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:06:16.517126Z",
     "start_time": "2019-06-01T04:06:16.509187Z"
    }
   },
   "outputs": [],
   "source": [
    "def load_and_prec():\n",
    "    train = pd.read_csv(TRAIN_DATA, index_col='id')\n",
    "    test = pd.read_csv(TEST_DATA, index_col='id')\n",
    "\n",
    "    # lower\n",
    "    train['comment_text'] = train['comment_text'].str.lower()\n",
    "    test['comment_text'] = test['comment_text'].str.lower()\n",
    "\n",
    "    # clean misspellings\n",
    "    train['comment_text'] = train['comment_text'].apply(replace_typical_misspell)\n",
    "    test['comment_text'] = test['comment_text'].apply(replace_typical_misspell)\n",
    "\n",
    "    # clean the text\n",
    "    train['comment_text'] = train['comment_text'].apply(clean_text)\n",
    "    test['comment_text'] = test['comment_text'].apply(clean_text)\n",
    "\n",
    "    # clean numbers\n",
    "    train['comment_text'] = train['comment_text'].apply(clean_numbers)\n",
    "    test['comment_text'] = test['comment_text'].apply(clean_numbers)\n",
    "    \n",
    "    # fill up the missing values\n",
    "    train_x = train['comment_text'].fillna('_##_').values\n",
    "    test_x = test['comment_text'].fillna('_##_').values\n",
    "    \n",
    "    # tokenize the sentences\n",
    "    tokenizer = Tokenizer(num_words=max_features)\n",
    "    tokenizer.fit_on_texts(list(train_x))\n",
    "    train_x = tokenizer.texts_to_sequences(train_x)\n",
    "    test_x = tokenizer.texts_to_sequences(test_x)\n",
    "\n",
    "    # pad the sentences\n",
    "    train_x = pad_sequences(train_x, maxlen=maxlen)\n",
    "    test_x = pad_sequences(test_x, maxlen=maxlen)\n",
    "    \n",
    "    # get the target values\n",
    "    identity_columns = [\n",
    "        'male', 'female', 'homosexual_gay_or_lesbian', 'christian', 'jewish',\n",
    "        'muslim', 'black', 'white', 'psychiatric_or_mental_illness']\n",
    "    train_y = (train['target'].values > 0.5).astype(int)\n",
    "    train_y_identity = train[identity_columns].values\n",
    "\n",
    "    # shuffling the data\n",
    "    np.random.seed(seed)\n",
    "    train_idx = np.random.permutation(len(train_x))\n",
    "\n",
    "    train_x = train_x[train_idx]\n",
    "    train_y = train_y[train_idx]\n",
    "    train_y_identity = train_y_identity[train_idx]\n",
    "\n",
    "    return train_x, train_y, train_y_identity, test_x, tokenizer.word_index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Preprocessing Pipline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:09:33.349840Z",
     "start_time": "2019-06-01T04:06:18.029737Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\75462\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\numpy\\lib\\arraysetops.py:569: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison\n",
      "  mask |= (ar1 == a)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 3min 15s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "identity_columns = [\n",
    "    'male', 'female', 'homosexual_gay_or_lesbian', 'christian', 'jewish',\n",
    "    'muslim', 'black', 'white', 'psychiatric_or_mental_illness']\n",
    "aux_columns = ['target', 'severe_toxicity', 'obscene', 'identity_attack', 'insult', 'threat']\n",
    "\n",
    "\n",
    "train = pd.read_csv(TRAIN_DATA, index_col='id')\n",
    "test = pd.read_csv(TEST_DATA, index_col='id')\n",
    "\n",
    "\n",
    "# Convert identity columns to booleans\n",
    "train = convert_dataframe_to_bool(train)\n",
    "\n",
    "# lower\n",
    "train['comment_text'] = train['comment_text'].str.lower()\n",
    "test['comment_text'] = test['comment_text'].str.lower()\n",
    "\n",
    "# clean misspellings\n",
    "train['comment_text'] = train['comment_text'].apply(replace_typical_misspell)\n",
    "test['comment_text'] = test['comment_text'].apply(replace_typical_misspell)\n",
    "\n",
    "# clean the text\n",
    "train['comment_text'] = train['comment_text'].apply(clean_text)\n",
    "test['comment_text'] = test['comment_text'].apply(clean_text)\n",
    "\n",
    "# clean numbers\n",
    "train['comment_text'] = train['comment_text'].apply(clean_numbers)\n",
    "test['comment_text'] = test['comment_text'].apply(clean_numbers)\n",
    "\n",
    "# fill up the missing values\n",
    "train['comment_text'] = train['comment_text'].fillna('_##_').values\n",
    "test['comment_text'] = test['comment_text'].fillna('_##_').values\n",
    "\n",
    "# tokenize the sentences\n",
    "tokenizer = Tokenizer(num_words=max_features)\n",
    "tokenizer.fit_on_texts(list(train['comment_text']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:09:33.459514Z",
     "start_time": "2019-06-01T04:09:33.351835Z"
    }
   },
   "outputs": [],
   "source": [
    "# get the target values\n",
    "train['target'] = (train['target'].values > 0.5).astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Statistic Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:11:12.088004Z",
     "start_time": "2019-06-01T04:09:33.461510Z"
    }
   },
   "outputs": [],
   "source": [
    "train['total_length'] = train['comment_text'].apply(len)\n",
    "train['capitals'] = train['comment_text'].apply(lambda comment: sum(1 for c in comment if c.isupper()))\n",
    "train['caps_vs_length'] = train.apply(lambda row: float(row['capitals'])/float(row['total_length']),axis=1)\n",
    "train['num_exclamation_marks'] = train['comment_text'].apply(lambda comment: comment.count('!'))\n",
    "train['num_question_marks'] = train['comment_text'].apply(lambda comment: comment.count('?'))\n",
    "train['num_punctuation'] = train['comment_text'].apply(lambda comment: sum(comment.count(w) for w in '.,;:'))\n",
    "train['num_symbols'] = train['comment_text'].apply(lambda comment: sum(comment.count(w) for w in '*&$%'))\n",
    "train['num_words'] = train['comment_text'].apply(lambda comment: len(comment.split()))\n",
    "train['num_unique_words'] = train['comment_text'].apply(lambda comment: len(set(w for w in comment.split())))\n",
    "train['words_vs_unique'] = train['num_unique_words'] / train['num_words']\n",
    "train['num_smilies'] = train['comment_text'].apply(lambda comment: sum(comment.count(w) for w in (':-)', ':)', ';-)', ';)')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:11:17.571438Z",
     "start_time": "2019-06-01T04:11:12.089965Z"
    }
   },
   "outputs": [],
   "source": [
    "test['total_length'] = test['comment_text'].apply(len)\n",
    "test['capitals'] = test['comment_text'].apply(lambda comment: sum(1 for c in comment if c.isupper()))\n",
    "test['caps_vs_length'] = test.apply(lambda row: float(row['capitals'])/float(row['total_length']),axis=1)\n",
    "test['num_exclamation_marks'] = test['comment_text'].apply(lambda comment: comment.count('!'))\n",
    "test['num_question_marks'] = test['comment_text'].apply(lambda comment: comment.count('?'))\n",
    "test['num_punctuation'] = test['comment_text'].apply(lambda comment: sum(comment.count(w) for w in '.,;:'))\n",
    "test['num_symbols'] = test['comment_text'].apply(lambda comment: sum(comment.count(w) for w in '*&$%'))\n",
    "test['num_words'] = test['comment_text'].apply(lambda comment: len(comment.split()))\n",
    "test['num_unique_words'] = test['comment_text'].apply(lambda comment: len(set(w for w in comment.split())))\n",
    "test['words_vs_unique'] = test['num_unique_words'] / test['num_words']\n",
    "test['num_smilies'] = test['comment_text'].apply(lambda comment: sum(comment.count(w) for w in (':-)', ':)', ';-)', ';)')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:11:17.576301Z",
     "start_time": "2019-06-01T04:11:17.572312Z"
    }
   },
   "outputs": [],
   "source": [
    "statistic_columns = \\\n",
    "['total_length','capitals','caps_vs_length','num_exclamation_marks',\\\n",
    " 'num_question_marks','num_punctuation','num_symbols','num_words','num_unique_words','words_vs_unique','num_smilies']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standardize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:11:17.588388Z",
     "start_time": "2019-06-01T04:11:17.582284Z"
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import Normalizer\n",
    "scaler = Normalizer()\n",
    "train[statistic_columns] = train[statistic_columns].fillna(0)\n",
    "test[statistic_columns] =test[statistic_columns].fillna(0)\n",
    "scaler.fit(train[statistic_columns])\n",
    "train[statistic_columns] = scaler.transform(train[statistic_columns])\n",
    "test[statistic_columns] = scaler.transform(test[statistic_columns])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Valid Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:11:30.559690Z",
     "start_time": "2019-06-01T04:11:21.376143Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1696581 train comments, 108293 validate comments\n"
     ]
    }
   ],
   "source": [
    "# split out validation set\n",
    "train, valid = train_test_split(train, test_size=0.06,random_state=2333,stratify=\\\n",
    "                                np.vstack([np.any(train[identity_columns],axis=1),train['target']]).T)\n",
    "print('%d train comments, %d validate comments' % (len(train), len(valid)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sorting TrainSet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:28.216661Z",
     "start_time": "2019-06-01T04:11:30.560688Z"
    }
   },
   "outputs": [],
   "source": [
    "train_x = tokenizer.texts_to_sequences(train['comment_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:29.257890Z",
     "start_time": "2019-06-01T04:12:28.217613Z"
    }
   },
   "outputs": [],
   "source": [
    "train['length'] =np.array([len(sentence) for sentence in train_x])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:30.608329Z",
     "start_time": "2019-06-01T04:12:29.262860Z"
    }
   },
   "outputs": [],
   "source": [
    "sorted_train = train.sort_values(by=['length'],ascending=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:30.762843Z",
     "start_time": "2019-06-01T04:12:30.610219Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target</th>\n",
       "      <th>comment_text</th>\n",
       "      <th>severe_toxicity</th>\n",
       "      <th>obscene</th>\n",
       "      <th>identity_attack</th>\n",
       "      <th>insult</th>\n",
       "      <th>threat</th>\n",
       "      <th>asian</th>\n",
       "      <th>atheist</th>\n",
       "      <th>bisexual</th>\n",
       "      <th>black</th>\n",
       "      <th>buddhist</th>\n",
       "      <th>christian</th>\n",
       "      <th>female</th>\n",
       "      <th>heterosexual</th>\n",
       "      <th>hindu</th>\n",
       "      <th>homosexual_gay_or_lesbian</th>\n",
       "      <th>intellectual_or_learning_disability</th>\n",
       "      <th>jewish</th>\n",
       "      <th>latino</th>\n",
       "      <th>male</th>\n",
       "      <th>muslim</th>\n",
       "      <th>other_disability</th>\n",
       "      <th>other_gender</th>\n",
       "      <th>other_race_or_ethnicity</th>\n",
       "      <th>other_religion</th>\n",
       "      <th>other_sexual_orientation</th>\n",
       "      <th>physical_disability</th>\n",
       "      <th>psychiatric_or_mental_illness</th>\n",
       "      <th>transgender</th>\n",
       "      <th>white</th>\n",
       "      <th>created_date</th>\n",
       "      <th>publication_id</th>\n",
       "      <th>parent_id</th>\n",
       "      <th>article_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>funny</th>\n",
       "      <th>wow</th>\n",
       "      <th>sad</th>\n",
       "      <th>likes</th>\n",
       "      <th>disagree</th>\n",
       "      <th>sexual_explicit</th>\n",
       "      <th>identity_annotator_count</th>\n",
       "      <th>toxicity_annotator_count</th>\n",
       "      <th>total_length</th>\n",
       "      <th>capitals</th>\n",
       "      <th>caps_vs_length</th>\n",
       "      <th>num_exclamation_marks</th>\n",
       "      <th>num_question_marks</th>\n",
       "      <th>num_punctuation</th>\n",
       "      <th>num_symbols</th>\n",
       "      <th>num_words</th>\n",
       "      <th>num_unique_words</th>\n",
       "      <th>words_vs_unique</th>\n",
       "      <th>num_smilies</th>\n",
       "      <th>length</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5444896</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-20 10:38:11.305896+00</td>\n",
       "      <td>54</td>\n",
       "      <td>5442386.0</td>\n",
       "      <td>345960</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961069</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192214</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192214</td>\n",
       "      <td>0.048053</td>\n",
       "      <td>0.012013</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>390222</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-07-16 16:48:41.188958+00</td>\n",
       "      <td>21</td>\n",
       "      <td>390062.0</td>\n",
       "      <td>141486</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>555273</th>\n",
       "      <td>0</td>\n",
       "      <td>;    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-26 22:59:27.084162+00</td>\n",
       "      <td>53</td>\n",
       "      <td>555108.0</td>\n",
       "      <td>149566</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5167560</th>\n",
       "      <td>0</td>\n",
       "      <td>-       -      !    !    !</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-23 12:34:52.471319+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328976</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0.981872</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.092051</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.153418</td>\n",
       "      <td>0.061367</td>\n",
       "      <td>0.012273</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1825271</th>\n",
       "      <td>0</td>\n",
       "      <td>;    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-06 21:52:16.791716+00</td>\n",
       "      <td>53</td>\n",
       "      <td>1085729.0</td>\n",
       "      <td>317162</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>595083</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-11-12 05:19:45.870267+00</td>\n",
       "      <td>22</td>\n",
       "      <td>595008.0</td>\n",
       "      <td>151364</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5167671</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-23 13:14:17.691301+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>329025</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>716706</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-16 04:39:54.372746+00</td>\n",
       "      <td>54</td>\n",
       "      <td>674299.0</td>\n",
       "      <td>155130</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6216305</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-10-24 23:29:28.346830+00</td>\n",
       "      <td>55</td>\n",
       "      <td>6216208.0</td>\n",
       "      <td>392936</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5347758</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-02 20:17:29.620335+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>340193</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>725142</th>\n",
       "      <td>0</td>\n",
       "      <td>-     .</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-18 06:11:24.966813+00</td>\n",
       "      <td>21</td>\n",
       "      <td>724958.0</td>\n",
       "      <td>157225</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.966988</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.080582</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.161165</td>\n",
       "      <td>0.161165</td>\n",
       "      <td>0.080582</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5827753</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-23 21:31:42.009867+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>369873</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>761091</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-28 02:13:11.181546+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>158807</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5086863</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-03 14:46:48.648776+00</td>\n",
       "      <td>54</td>\n",
       "      <td>5086734.0</td>\n",
       "      <td>324119</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5422919</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-15 19:52:00.255718+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>344838</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>291159</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    (</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-04-21 20:23:02.814628+00</td>\n",
       "      <td>21</td>\n",
       "      <td>NaN</td>\n",
       "      <td>63244</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6257428</th>\n",
       "      <td>0</td>\n",
       "      <td>=    (</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-10-31 12:14:28.900997+00</td>\n",
       "      <td>102</td>\n",
       "      <td>NaN</td>\n",
       "      <td>395334</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>0.957826</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191565</td>\n",
       "      <td>0.191565</td>\n",
       "      <td>0.095783</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5877618</th>\n",
       "      <td>0</td>\n",
       "      <td>.     %</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-01 02:51:58.022572+00</td>\n",
       "      <td>55</td>\n",
       "      <td>5876417.0</td>\n",
       "      <td>372881</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.963863</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.080322</td>\n",
       "      <td>0.080322</td>\n",
       "      <td>0.160644</td>\n",
       "      <td>0.160644</td>\n",
       "      <td>0.080322</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>708892</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    )    .</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-14 16:43:12.928227+00</td>\n",
       "      <td>53</td>\n",
       "      <td>708792.0</td>\n",
       "      <td>156498</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.956730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095673</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536617</th>\n",
       "      <td>0</td>\n",
       "      <td>;    -    )    )    )    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-19 04:12:39.477289+00</td>\n",
       "      <td>21</td>\n",
       "      <td>NaN</td>\n",
       "      <td>148904</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.975255</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.032509</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.195051</td>\n",
       "      <td>0.097526</td>\n",
       "      <td>0.016254</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5637985</th>\n",
       "      <td>0</td>\n",
       "      <td>&lt;</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-22 06:00:36.142877+00</td>\n",
       "      <td>21</td>\n",
       "      <td>NaN</td>\n",
       "      <td>358064</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5690001</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-31 15:07:13.856649+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5689946.0</td>\n",
       "      <td>360726</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.956730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.095673</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>383654</th>\n",
       "      <td>0</td>\n",
       "      <td>;    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2015-11-17 20:56:30.180896+00</td>\n",
       "      <td>25</td>\n",
       "      <td>383653.0</td>\n",
       "      <td>141164</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6027277</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-26 16:42:59.989320+00</td>\n",
       "      <td>102</td>\n",
       "      <td>6027205.0</td>\n",
       "      <td>381959</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.956730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.095673</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>406194</th>\n",
       "      <td>0</td>\n",
       "      <td>:    (</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-07-28 23:39:54.828646+00</td>\n",
       "      <td>21</td>\n",
       "      <td>NaN</td>\n",
       "      <td>142541</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>565755</th>\n",
       "      <td>0</td>\n",
       "      <td>*     .     %</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-31 20:48:55.432166+00</td>\n",
       "      <td>21</td>\n",
       "      <td>565194.0</td>\n",
       "      <td>149977</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960897</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.056523</td>\n",
       "      <td>0.113047</td>\n",
       "      <td>0.169570</td>\n",
       "      <td>0.169570</td>\n",
       "      <td>0.056523</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>811889</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-01-09 09:29:48.577420+00</td>\n",
       "      <td>21</td>\n",
       "      <td>809924.0</td>\n",
       "      <td>160345</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6179046</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-10-19 14:00:02.507570+00</td>\n",
       "      <td>54</td>\n",
       "      <td>6178961.0</td>\n",
       "      <td>390870</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5033672</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-21 15:22:25.098829+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>321032</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5341367</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-01 20:11:07.496822+00</td>\n",
       "      <td>13</td>\n",
       "      <td>5340683.0</td>\n",
       "      <td>339490</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5415652</th>\n",
       "      <td>0</td>\n",
       "      <td>.    .    .</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-14 20:23:56.757244+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5409754.0</td>\n",
       "      <td>343874</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960058</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.064004</td>\n",
       "      <td>0.021335</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5074917</th>\n",
       "      <td>0</td>\n",
       "      <td>:    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-30 20:45:19.115199+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5074869.0</td>\n",
       "      <td>322741</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5808050</th>\n",
       "      <td>0</td>\n",
       "      <td>:    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-20 21:22:50.497601+00</td>\n",
       "      <td>102</td>\n",
       "      <td>5806418.0</td>\n",
       "      <td>368647</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5040012</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-22 19:44:52.389174+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>321393</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1063388</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-01 22:48:03.314604+00</td>\n",
       "      <td>54</td>\n",
       "      <td>1063381.0</td>\n",
       "      <td>316099</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960058</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.064004</td>\n",
       "      <td>0.021335</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5677817</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-28 19:50:51.754377+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5676553.0</td>\n",
       "      <td>360154</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961510</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192302</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192302</td>\n",
       "      <td>0.038460</td>\n",
       "      <td>0.007692</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>313335</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-05-13 20:26:09.283360+00</td>\n",
       "      <td>6</td>\n",
       "      <td>313228.0</td>\n",
       "      <td>75668</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961069</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192214</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192214</td>\n",
       "      <td>0.048053</td>\n",
       "      <td>0.012013</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5882124</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-01 21:57:30.507680+00</td>\n",
       "      <td>55</td>\n",
       "      <td>5877936.0</td>\n",
       "      <td>372887</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961510</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192302</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192302</td>\n",
       "      <td>0.038460</td>\n",
       "      <td>0.007692</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5312539</th>\n",
       "      <td>0</td>\n",
       "      <td>+</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-05-27 11:59:04.576758+00</td>\n",
       "      <td>54</td>\n",
       "      <td>5304935.0</td>\n",
       "      <td>337670</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>872483</th>\n",
       "      <td>0</td>\n",
       "      <td>;    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-01-21 03:55:32.479786+00</td>\n",
       "      <td>21</td>\n",
       "      <td>872436.0</td>\n",
       "      <td>163011</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.967096</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.056888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.170664</td>\n",
       "      <td>0.170664</td>\n",
       "      <td>0.056888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>549670</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-24 23:05:38.896358+00</td>\n",
       "      <td>21</td>\n",
       "      <td>548609.0</td>\n",
       "      <td>149406</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>675443</th>\n",
       "      <td>0</td>\n",
       "      <td>.     +</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-07 18:59:32.316744+00</td>\n",
       "      <td>53</td>\n",
       "      <td>675104.0</td>\n",
       "      <td>155136</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.966988</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.080582</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.161165</td>\n",
       "      <td>0.161165</td>\n",
       "      <td>0.080582</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5458446</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-22 14:58:30.945327+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>347350</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5396549</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-11 17:20:39.870366+00</td>\n",
       "      <td>102</td>\n",
       "      <td>5396260.0</td>\n",
       "      <td>343102</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>357500</th>\n",
       "      <td>0</td>\n",
       "      <td>%</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-06-20 05:05:54.672941+00</td>\n",
       "      <td>21</td>\n",
       "      <td>357436.0</td>\n",
       "      <td>139153</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.948683</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>973919</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-02-09 21:14:37.098612+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>167198</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6093418</th>\n",
       "      <td>0</td>\n",
       "      <td>,     .</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-10-06 04:10:33.278544+00</td>\n",
       "      <td>55</td>\n",
       "      <td>6092401.0</td>\n",
       "      <td>385792</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.957704</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.159617</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.159617</td>\n",
       "      <td>0.159617</td>\n",
       "      <td>0.079809</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>610989</th>\n",
       "      <td>0</td>\n",
       "      <td>:    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-11-18 18:19:38.339743+00</td>\n",
       "      <td>53</td>\n",
       "      <td>610955.0</td>\n",
       "      <td>152181</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5706538</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-03 03:53:57.612378+00</td>\n",
       "      <td>43</td>\n",
       "      <td>5678200.0</td>\n",
       "      <td>360313</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961524</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.137361</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.137361</td>\n",
       "      <td>0.137361</td>\n",
       "      <td>0.137361</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5466950</th>\n",
       "      <td>0</td>\n",
       "      <td>:    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-23 19:40:54.981434+00</td>\n",
       "      <td>13</td>\n",
       "      <td>5466941.0</td>\n",
       "      <td>347455</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>693656</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?    ?    ?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-11 15:23:35.049818+00</td>\n",
       "      <td>54</td>\n",
       "      <td>684562.0</td>\n",
       "      <td>155499</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.962028</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192406</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192406</td>\n",
       "      <td>0.021378</td>\n",
       "      <td>0.002375</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6322637</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-11-09 16:37:32.919073+00</td>\n",
       "      <td>54</td>\n",
       "      <td>6321955.0</td>\n",
       "      <td>398850</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5772784</th>\n",
       "      <td>0</td>\n",
       "      <td>+</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-15 16:47:42.475821+00</td>\n",
       "      <td>102</td>\n",
       "      <td>5771399.0</td>\n",
       "      <td>366454</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>600543</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-11-14 20:59:05.669127+00</td>\n",
       "      <td>22</td>\n",
       "      <td>NaN</td>\n",
       "      <td>151608</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5134770</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-14 23:29:02.122797+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5134753.0</td>\n",
       "      <td>324719</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.956730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.095673</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5525762</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-04 14:49:28.945132+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>351590</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5905439</th>\n",
       "      <td>0</td>\n",
       "      <td>+</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-06 07:13:57.380740+00</td>\n",
       "      <td>82</td>\n",
       "      <td>NaN</td>\n",
       "      <td>374576</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5512374</th>\n",
       "      <td>0</td>\n",
       "      <td>-    -</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-01 20:41:37.504062+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5510353.0</td>\n",
       "      <td>350679</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>0.974740</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.194948</td>\n",
       "      <td>0.097474</td>\n",
       "      <td>0.048737</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>521788</th>\n",
       "      <td>0</td>\n",
       "      <td>;    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-11 17:08:29.507636+00</td>\n",
       "      <td>53</td>\n",
       "      <td>521773.0</td>\n",
       "      <td>148116</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>775130</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-31 20:23:38.078872+00</td>\n",
       "      <td>53</td>\n",
       "      <td>771259.0</td>\n",
       "      <td>159232</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0.956730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.095673</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>909742</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-01-27 23:01:35.878519+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>164554</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>0.948683</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5041844</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-23 02:19:55.123133+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5040120.0</td>\n",
       "      <td>321351</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6053985</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-30 00:44:15.488260+00</td>\n",
       "      <td>55</td>\n",
       "      <td>6053800.0</td>\n",
       "      <td>383662</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>653203</th>\n",
       "      <td>0</td>\n",
       "      <td>+       +       =       +       +       =</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-03 00:44:29.598338+00</td>\n",
       "      <td>21</td>\n",
       "      <td>652922.0</td>\n",
       "      <td>153859</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.991750</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.121439</td>\n",
       "      <td>0.040480</td>\n",
       "      <td>0.006747</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5768054</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-14 21:18:37.365002+00</td>\n",
       "      <td>54</td>\n",
       "      <td>5767387.0</td>\n",
       "      <td>366444</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961880</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192376</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192376</td>\n",
       "      <td>0.027482</td>\n",
       "      <td>0.003926</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>541868</th>\n",
       "      <td>0</td>\n",
       "      <td>+</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-21 04:05:28.403972+00</td>\n",
       "      <td>13</td>\n",
       "      <td>NaN</td>\n",
       "      <td>149049</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5309075</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-05-26 17:21:17.680200+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>337957</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5208691</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-05-03 12:18:49.074999+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>331560</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5699587</th>\n",
       "      <td>0</td>\n",
       "      <td>$    $    $    $    $    $    $</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-01 23:49:47.063278+00</td>\n",
       "      <td>55</td>\n",
       "      <td>NaN</td>\n",
       "      <td>361519</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961880</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192376</td>\n",
       "      <td>0.192376</td>\n",
       "      <td>0.027482</td>\n",
       "      <td>0.003926</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>558939</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-28 13:56:14.595471+00</td>\n",
       "      <td>53</td>\n",
       "      <td>558545.0</td>\n",
       "      <td>149770</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960058</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.064004</td>\n",
       "      <td>0.021335</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5143584</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-17 21:23:47.219202+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>327426</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5165981</th>\n",
       "      <td>0</td>\n",
       "      <td>:    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-22 20:53:36.965769+00</td>\n",
       "      <td>55</td>\n",
       "      <td>5164990.0</td>\n",
       "      <td>328849</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>566673</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-11-01 05:02:50.160294+00</td>\n",
       "      <td>53</td>\n",
       "      <td>566591.0</td>\n",
       "      <td>150110</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.956730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191346</td>\n",
       "      <td>0.095673</td>\n",
       "      <td>0.047836</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429233</th>\n",
       "      <td>0</td>\n",
       "      <td>:    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-08-13 04:08:32.951058+00</td>\n",
       "      <td>21</td>\n",
       "      <td>428947.0</td>\n",
       "      <td>143578</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.953463</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.190693</td>\n",
       "      <td>0.095346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5929020</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-10 14:45:24.023741+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>376161</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5207990</th>\n",
       "      <td>0</td>\n",
       "      <td>+</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-05-03 03:52:41.161333+00</td>\n",
       "      <td>55</td>\n",
       "      <td>5206937.0</td>\n",
       "      <td>331285</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6041871</th>\n",
       "      <td>0</td>\n",
       "      <td>%</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-28 14:58:31.753988+00</td>\n",
       "      <td>54</td>\n",
       "      <td>6032069.0</td>\n",
       "      <td>382382</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.948683</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.158114</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5905026</th>\n",
       "      <td>0</td>\n",
       "      <td>+     =</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-06 04:12:13.850119+00</td>\n",
       "      <td>55</td>\n",
       "      <td>NaN</td>\n",
       "      <td>374517</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.974391</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.149906</td>\n",
       "      <td>0.149906</td>\n",
       "      <td>0.074953</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5464111</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-06-23 12:27:23.965567+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>347728</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5779787</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-16 14:54:56.981743+00</td>\n",
       "      <td>102</td>\n",
       "      <td>5779414.0</td>\n",
       "      <td>366993</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961069</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192214</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192214</td>\n",
       "      <td>0.048053</td>\n",
       "      <td>0.012013</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>767492</th>\n",
       "      <td>0</td>\n",
       "      <td>(      *       /     .     )     -     (    ...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-29 21:22:55.803866+00</td>\n",
       "      <td>54</td>\n",
       "      <td>767025.0</td>\n",
       "      <td>159096</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0.983997</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.027718</td>\n",
       "      <td>0.027718</td>\n",
       "      <td>0.152450</td>\n",
       "      <td>0.083155</td>\n",
       "      <td>0.007560</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5756759</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-08-12 20:48:42.438363+00</td>\n",
       "      <td>54</td>\n",
       "      <td>5756126.0</td>\n",
       "      <td>365775</td>\n",
       "      <td>approved</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960058</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.064004</td>\n",
       "      <td>0.021335</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5689450</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-31 13:13:32.216426+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>361247</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1065839</th>\n",
       "      <td>0</td>\n",
       "      <td>.     %</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-03-02 15:03:03.813708+00</td>\n",
       "      <td>54</td>\n",
       "      <td>1065632.0</td>\n",
       "      <td>316260</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.963863</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.080322</td>\n",
       "      <td>0.080322</td>\n",
       "      <td>0.160644</td>\n",
       "      <td>0.160644</td>\n",
       "      <td>0.080322</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>907107</th>\n",
       "      <td>0</td>\n",
       "      <td>$     ,     !    !    !    !    !    !</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-01-27 16:20:14.455600+00</td>\n",
       "      <td>21</td>\n",
       "      <td>NaN</td>\n",
       "      <td>164311</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>0.969912</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.138559</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.023093</td>\n",
       "      <td>0.023093</td>\n",
       "      <td>0.184745</td>\n",
       "      <td>0.069279</td>\n",
       "      <td>0.008660</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>509921</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-06 02:18:31.698833+00</td>\n",
       "      <td>21</td>\n",
       "      <td>508981.0</td>\n",
       "      <td>147620</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>676325</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-12-07 21:19:42.647097+00</td>\n",
       "      <td>21</td>\n",
       "      <td>675879.0</td>\n",
       "      <td>155064</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>517741</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2016-10-09 18:55:39.342745+00</td>\n",
       "      <td>53</td>\n",
       "      <td>513419.0</td>\n",
       "      <td>147885</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.960058</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192012</td>\n",
       "      <td>0.064004</td>\n",
       "      <td>0.021335</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1007954</th>\n",
       "      <td>0</td>\n",
       "      <td>/</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-02-17 16:39:28.241073+00</td>\n",
       "      <td>54</td>\n",
       "      <td>1007851.0</td>\n",
       "      <td>168840</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.970725</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.138675</td>\n",
       "      <td>0.138675</td>\n",
       "      <td>0.138675</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5528968</th>\n",
       "      <td>0</td>\n",
       "      <td>$    $    $    $    $    $    $    $    $   ...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-07-04 23:33:31.491559+00</td>\n",
       "      <td>55</td>\n",
       "      <td>NaN</td>\n",
       "      <td>351704</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>0.962159</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192432</td>\n",
       "      <td>0.192432</td>\n",
       "      <td>0.013745</td>\n",
       "      <td>0.000982</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5247193</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-05-12 11:35:57.537105+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>334058</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5129351</th>\n",
       "      <td>0</td>\n",
       "      <td>,</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-04-13 16:53:32.841972+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>326715</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5947689</th>\n",
       "      <td>0</td>\n",
       "      <td>.</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-13 19:51:37.548488+00</td>\n",
       "      <td>54</td>\n",
       "      <td>NaN</td>\n",
       "      <td>377417</td>\n",
       "      <td>rejected</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>927871</th>\n",
       "      <td>0</td>\n",
       "      <td>+</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-01-31 18:01:43.343070+00</td>\n",
       "      <td>54</td>\n",
       "      <td>927799.0</td>\n",
       "      <td>165222</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.960769</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.160128</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5967514</th>\n",
       "      <td>0</td>\n",
       "      <td>?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-17 02:31:48.389817+00</td>\n",
       "      <td>105</td>\n",
       "      <td>5967277.0</td>\n",
       "      <td>378445</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.928477</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.185695</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5323359</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-05-29 20:25:16.352274+00</td>\n",
       "      <td>53</td>\n",
       "      <td>5323293.0</td>\n",
       "      <td>336879</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6044052</th>\n",
       "      <td>0</td>\n",
       "      <td>?    ?    ?    ?    ?</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-09-28 19:46:22.770697+00</td>\n",
       "      <td>102</td>\n",
       "      <td>6033935.0</td>\n",
       "      <td>382359</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>0.961510</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192302</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.192302</td>\n",
       "      <td>0.038460</td>\n",
       "      <td>0.007692</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>852392</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-01-17 15:27:20.346711+00</td>\n",
       "      <td>53</td>\n",
       "      <td>NaN</td>\n",
       "      <td>162373</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6172470</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-10-18 17:31:34.963777+00</td>\n",
       "      <td>102</td>\n",
       "      <td>NaN</td>\n",
       "      <td>390595</td>\n",
       "      <td>approved</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6227710</th>\n",
       "      <td>0</td>\n",
       "      <td>:    -    )</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>2017-10-26 16:19:49.616084+00</td>\n",
       "      <td>54</td>\n",
       "      <td>6227476.0</td>\n",
       "      <td>393582</td>\n",
       "      <td>approved</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.958315</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.191663</td>\n",
       "      <td>0.063888</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         target                                       comment_text  \\\n",
       "id                                                                   \n",
       "5444896       0                                 ?    ?    ?    ?     \n",
       "390222        0                                                ?     \n",
       "555273        0                                      ;    -    )     \n",
       "5167560       0                       -       -      !    !    !     \n",
       "1825271       0                                      ;    -    )     \n",
       "595083        0                                                ?     \n",
       "5167671       0                                                .     \n",
       "716706        0                                                      \n",
       "6216305       0                                                      \n",
       "5347758       0                                                .     \n",
       "725142        0                                          -     .     \n",
       "5827753       0                                                .     \n",
       "761091        0                                                .     \n",
       "5086863       0                                                      \n",
       "5422919       0                                                .     \n",
       "291159        0                                      :    -    (     \n",
       "6257428       0                                           =    (     \n",
       "5877618       0                                          .     %     \n",
       "708892        0                                 :    -    )    .     \n",
       "536617        0                       ;    -    )    )    )    )     \n",
       "5637985       0                                               <      \n",
       "5690001       0                                           ?    ?     \n",
       "383654        0                                           ;    )     \n",
       "6027277       0                                           ?    ?     \n",
       "406194        0                                           :    (     \n",
       "565755        0                                    *     .     %     \n",
       "811889        0                                                ?     \n",
       "6179046       0                                                ?     \n",
       "5033672       0                                                .     \n",
       "5341367       0                                                ?     \n",
       "5415652       0                                      .    .    .     \n",
       "5074917       0                                           :    )     \n",
       "5808050       0                                           :    )     \n",
       "5040012       0                                                .     \n",
       "1063388       0                                      ?    ?    ?     \n",
       "5677817       0                            ?    ?    ?    ?    ?     \n",
       "313335        0                                 ?    ?    ?    ?     \n",
       "5882124       0                            ?    ?    ?    ?    ?     \n",
       "5312539       0                                               +      \n",
       "872483        0                                      ;    -    )     \n",
       "549670        0                                                ?     \n",
       "675443        0                                          .     +     \n",
       "5458446       0                                                ,     \n",
       "5396549       0                                                      \n",
       "357500        0                                                %     \n",
       "973919        0                                                      \n",
       "6093418       0                                          ,     .     \n",
       "610989        0                                           :    )     \n",
       "5706538       0                                               .      \n",
       "5466950       0                                           :    )     \n",
       "693656        0        ?    ?    ?    ?    ?    ?    ?    ?    ?     \n",
       "6322637       0                                                      \n",
       "5772784       0                                               +      \n",
       "600543        0                                                      \n",
       "5134770       0                                           ?    ?     \n",
       "5525762       0                                                .     \n",
       "5905439       0                                               +      \n",
       "5512374       0                                           -    -     \n",
       "521788        0                                      ;    -    )     \n",
       "775130        0                                           ?    ?     \n",
       "909742        0                                                .     \n",
       "5041844       0                                      :    -    )     \n",
       "6053985       0                                                ?     \n",
       "653203        0      +       +       =       +       +       =       \n",
       "5768054       0                  ?    ?    ?    ?    ?    ?    ?     \n",
       "541868        0                                               +      \n",
       "5309075       0                                                ,     \n",
       "5208691       0                                                ,     \n",
       "5699587       0                  $    $    $    $    $    $    $     \n",
       "558939        0                                      ?    ?    ?     \n",
       "5143584       0                                                      \n",
       "5165981       0                                           :    )     \n",
       "566673        0                                           ?    ?     \n",
       "429233        0                                           :    )     \n",
       "5929020       0                                                .     \n",
       "5207990       0                                               +      \n",
       "6041871       0                                                %     \n",
       "5905026       0                                         +     =      \n",
       "5464111       0                                                ,     \n",
       "5779787       0                                 ?    ?    ?    ?     \n",
       "767492        0    (      *       /     .     )     -     (    ...   \n",
       "5756759       0                                      ?    ?    ?     \n",
       "5689450       0                                                ,     \n",
       "1065839       0                                          .     %     \n",
       "907107        0           $     ,     !    !    !    !    !    !     \n",
       "509921        0                                                ?     \n",
       "676325        0                                                      \n",
       "517741        0                                      ?    ?    ?     \n",
       "1007954       0                                               /      \n",
       "5528968       0    $    $    $    $    $    $    $    $    $   ...   \n",
       "5247193       0                                                ,     \n",
       "5129351       0                                                ,     \n",
       "5947689       0                                                .     \n",
       "927871        0                                               +      \n",
       "5967514       0                                                ?     \n",
       "5323359       0                                      :    -    )     \n",
       "6044052       0                            ?    ?    ?    ?    ?     \n",
       "852392        0                                      :    -    )     \n",
       "6172470       0                                      :    -    )     \n",
       "6227710       0                                      :    -    )     \n",
       "\n",
       "         severe_toxicity  obscene  identity_attack  insult  threat  asian  \\\n",
       "id                                                                          \n",
       "5444896              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "390222               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "555273               0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "5167560              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "1825271              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "595083               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5167671              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "716706               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6216305              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5347758              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "725142               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5827753              0.0      0.0              0.0     0.1     0.0    NaN   \n",
       "761091               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5086863              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5422919              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "291159               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6257428              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "5877618              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "708892               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "536617               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5637985              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "5690001              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "383654               0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "6027277              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "406194               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "565755               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "811889               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6179046              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5033672              0.0      0.0              0.0     0.1     0.0    NaN   \n",
       "5341367              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5415652              0.0      0.0              0.0     0.1     0.1    NaN   \n",
       "5074917              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5808050              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5040012              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "1063388              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5677817              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "313335               0.0      0.0              0.1     0.1     0.0    NaN   \n",
       "5882124              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5312539              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "872483               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "549670               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "675443               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5458446              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5396549              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "357500               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "973919               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6093418              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "610989               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5706538              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5466950              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "693656               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6322637              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5772784              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "600543               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5134770              0.0      0.0              0.1     0.1     0.0    NaN   \n",
       "5525762              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5905439              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5512374              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "521788               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "775130               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "909742               0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "5041844              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6053985              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "653203               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5768054              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "541868               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5309075              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5208691              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5699587              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "558939               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5143584              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5165981              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "566673               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "429233               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5929020              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5207990              0.0      0.0              0.0     0.1     0.0    NaN   \n",
       "6041871              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5905026              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5464111              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5779787              0.0      0.0              0.1     0.1     0.0    NaN   \n",
       "767492               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5756759              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5689450              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "1065839              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "907107               0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "509921               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "676325               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "517741               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "1007954              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5528968              0.0      0.0              0.0     0.0     0.0    0.0   \n",
       "5247193              0.0      0.0              0.0     0.1     0.1    NaN   \n",
       "5129351              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5947689              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "927871               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5967514              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "5323359              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6044052              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "852392               0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6172470              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "6227710              0.0      0.0              0.0     0.0     0.0    NaN   \n",
       "\n",
       "         atheist  bisexual  black  buddhist  christian  female  heterosexual  \\\n",
       "id                                                                             \n",
       "5444896      NaN       NaN  False       NaN      False   False           NaN   \n",
       "390222       NaN       NaN  False       NaN      False   False           NaN   \n",
       "555273       0.0       0.0  False       0.0      False   False           0.0   \n",
       "5167560      NaN       NaN  False       NaN      False   False           NaN   \n",
       "1825271      NaN       NaN  False       NaN      False   False           NaN   \n",
       "595083       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5167671      NaN       NaN  False       NaN      False   False           NaN   \n",
       "716706       NaN       NaN  False       NaN      False   False           NaN   \n",
       "6216305      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5347758      NaN       NaN  False       NaN      False   False           NaN   \n",
       "725142       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5827753      NaN       NaN  False       NaN      False   False           NaN   \n",
       "761091       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5086863      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5422919      NaN       NaN  False       NaN      False   False           NaN   \n",
       "291159       NaN       NaN  False       NaN      False   False           NaN   \n",
       "6257428      0.0       0.0  False       0.0      False   False           0.0   \n",
       "5877618      NaN       NaN  False       NaN      False   False           NaN   \n",
       "708892       NaN       NaN  False       NaN      False   False           NaN   \n",
       "536617       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5637985      0.0       0.0  False       0.0      False   False           0.0   \n",
       "5690001      NaN       NaN  False       NaN      False   False           NaN   \n",
       "383654       0.0       0.0  False       0.0      False   False           0.0   \n",
       "6027277      NaN       NaN  False       NaN      False   False           NaN   \n",
       "406194       NaN       NaN  False       NaN      False   False           NaN   \n",
       "565755       NaN       NaN  False       NaN      False   False           NaN   \n",
       "811889       NaN       NaN  False       NaN      False   False           NaN   \n",
       "6179046      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5033672      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5341367      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5415652      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5074917      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5808050      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5040012      NaN       NaN  False       NaN      False   False           NaN   \n",
       "1063388      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5677817      NaN       NaN  False       NaN      False   False           NaN   \n",
       "313335       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5882124      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5312539      0.0       0.0  False       0.0      False   False           0.0   \n",
       "872483       NaN       NaN  False       NaN      False   False           NaN   \n",
       "549670       NaN       NaN  False       NaN      False   False           NaN   \n",
       "675443       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5458446      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5396549      NaN       NaN  False       NaN      False   False           NaN   \n",
       "357500       NaN       NaN  False       NaN      False   False           NaN   \n",
       "973919       NaN       NaN  False       NaN      False   False           NaN   \n",
       "6093418      NaN       NaN  False       NaN      False   False           NaN   \n",
       "610989       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5706538      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5466950      NaN       NaN  False       NaN      False   False           NaN   \n",
       "693656       NaN       NaN  False       NaN      False   False           NaN   \n",
       "6322637      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5772784      NaN       NaN  False       NaN      False   False           NaN   \n",
       "600543       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5134770      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5525762      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5905439      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5512374      0.0       0.0  False       0.0      False   False           0.0   \n",
       "521788       NaN       NaN  False       NaN      False   False           NaN   \n",
       "775130       NaN       NaN  False       NaN      False   False           NaN   \n",
       "909742       0.0       0.0  False       0.0      False   False           0.0   \n",
       "5041844      NaN       NaN  False       NaN      False   False           NaN   \n",
       "6053985      0.0       0.0  False       0.0      False   False           0.0   \n",
       "653203       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5768054      NaN       NaN  False       NaN      False   False           NaN   \n",
       "541868       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5309075      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5208691      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5699587      NaN       NaN  False       NaN      False   False           NaN   \n",
       "558939       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5143584      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5165981      0.0       0.0  False       0.0      False   False           0.0   \n",
       "566673       NaN       NaN  False       NaN      False   False           NaN   \n",
       "429233       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5929020      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5207990      NaN       NaN  False       NaN      False   False           NaN   \n",
       "6041871      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5905026      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5464111      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5779787      NaN       NaN  False       NaN      False   False           NaN   \n",
       "767492       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5756759      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5689450      NaN       NaN  False       NaN      False   False           NaN   \n",
       "1065839      NaN       NaN  False       NaN      False   False           NaN   \n",
       "907107       0.0       0.0  False       0.0      False   False           0.0   \n",
       "509921       NaN       NaN  False       NaN      False   False           NaN   \n",
       "676325       NaN       NaN  False       NaN      False   False           NaN   \n",
       "517741       NaN       NaN  False       NaN      False   False           NaN   \n",
       "1007954      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5528968      0.0       0.0  False       0.0      False   False           0.0   \n",
       "5247193      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5129351      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5947689      NaN       NaN  False       NaN      False   False           NaN   \n",
       "927871       NaN       NaN  False       NaN      False   False           NaN   \n",
       "5967514      NaN       NaN  False       NaN      False   False           NaN   \n",
       "5323359      NaN       NaN  False       NaN      False   False           NaN   \n",
       "6044052      NaN       NaN  False       NaN      False   False           NaN   \n",
       "852392       NaN       NaN  False       NaN      False   False           NaN   \n",
       "6172470      NaN       NaN  False       NaN      False   False           NaN   \n",
       "6227710      NaN       NaN  False       NaN      False   False           NaN   \n",
       "\n",
       "         hindu  homosexual_gay_or_lesbian  \\\n",
       "id                                          \n",
       "5444896    NaN                      False   \n",
       "390222     NaN                      False   \n",
       "555273     0.0                      False   \n",
       "5167560    NaN                      False   \n",
       "1825271    NaN                      False   \n",
       "595083     NaN                      False   \n",
       "5167671    NaN                      False   \n",
       "716706     NaN                      False   \n",
       "6216305    NaN                      False   \n",
       "5347758    NaN                      False   \n",
       "725142     NaN                      False   \n",
       "5827753    NaN                      False   \n",
       "761091     NaN                      False   \n",
       "5086863    NaN                      False   \n",
       "5422919    NaN                      False   \n",
       "291159     NaN                      False   \n",
       "6257428    0.0                      False   \n",
       "5877618    NaN                      False   \n",
       "708892     NaN                      False   \n",
       "536617     NaN                      False   \n",
       "5637985    0.0                      False   \n",
       "5690001    NaN                      False   \n",
       "383654     0.0                      False   \n",
       "6027277    NaN                      False   \n",
       "406194     NaN                      False   \n",
       "565755     NaN                      False   \n",
       "811889     NaN                      False   \n",
       "6179046    NaN                      False   \n",
       "5033672    NaN                      False   \n",
       "5341367    NaN                      False   \n",
       "5415652    NaN                      False   \n",
       "5074917    NaN                      False   \n",
       "5808050    NaN                      False   \n",
       "5040012    NaN                      False   \n",
       "1063388    NaN                      False   \n",
       "5677817    NaN                      False   \n",
       "313335     NaN                      False   \n",
       "5882124    NaN                      False   \n",
       "5312539    0.0                      False   \n",
       "872483     NaN                      False   \n",
       "549670     NaN                      False   \n",
       "675443     NaN                      False   \n",
       "5458446    NaN                      False   \n",
       "5396549    NaN                      False   \n",
       "357500     NaN                      False   \n",
       "973919     NaN                      False   \n",
       "6093418    NaN                      False   \n",
       "610989     NaN                      False   \n",
       "5706538    NaN                      False   \n",
       "5466950    NaN                      False   \n",
       "693656     NaN                      False   \n",
       "6322637    NaN                      False   \n",
       "5772784    NaN                      False   \n",
       "600543     NaN                      False   \n",
       "5134770    NaN                      False   \n",
       "5525762    NaN                      False   \n",
       "5905439    NaN                      False   \n",
       "5512374    0.0                      False   \n",
       "521788     NaN                      False   \n",
       "775130     NaN                      False   \n",
       "909742     0.0                      False   \n",
       "5041844    NaN                      False   \n",
       "6053985    0.0                      False   \n",
       "653203     NaN                      False   \n",
       "5768054    NaN                      False   \n",
       "541868     NaN                      False   \n",
       "5309075    NaN                      False   \n",
       "5208691    NaN                      False   \n",
       "5699587    NaN                      False   \n",
       "558939     NaN                      False   \n",
       "5143584    NaN                      False   \n",
       "5165981    0.0                      False   \n",
       "566673     NaN                      False   \n",
       "429233     NaN                      False   \n",
       "5929020    NaN                      False   \n",
       "5207990    NaN                      False   \n",
       "6041871    NaN                      False   \n",
       "5905026    NaN                      False   \n",
       "5464111    NaN                      False   \n",
       "5779787    NaN                      False   \n",
       "767492     NaN                      False   \n",
       "5756759    NaN                      False   \n",
       "5689450    NaN                      False   \n",
       "1065839    NaN                      False   \n",
       "907107     0.0                      False   \n",
       "509921     NaN                      False   \n",
       "676325     NaN                      False   \n",
       "517741     NaN                      False   \n",
       "1007954    NaN                      False   \n",
       "5528968    0.0                      False   \n",
       "5247193    NaN                      False   \n",
       "5129351    NaN                      False   \n",
       "5947689    NaN                      False   \n",
       "927871     NaN                      False   \n",
       "5967514    NaN                      False   \n",
       "5323359    NaN                      False   \n",
       "6044052    NaN                      False   \n",
       "852392     NaN                      False   \n",
       "6172470    NaN                      False   \n",
       "6227710    NaN                      False   \n",
       "\n",
       "         intellectual_or_learning_disability  jewish  latino   male  muslim  \\\n",
       "id                                                                            \n",
       "5444896                                  NaN   False     NaN  False   False   \n",
       "390222                                   NaN   False     NaN  False   False   \n",
       "555273                                   0.0   False     0.0  False   False   \n",
       "5167560                                  NaN   False     NaN  False   False   \n",
       "1825271                                  NaN   False     NaN  False   False   \n",
       "595083                                   NaN   False     NaN  False   False   \n",
       "5167671                                  NaN   False     NaN  False   False   \n",
       "716706                                   NaN   False     NaN  False   False   \n",
       "6216305                                  NaN   False     NaN  False   False   \n",
       "5347758                                  NaN   False     NaN  False   False   \n",
       "725142                                   NaN   False     NaN  False   False   \n",
       "5827753                                  NaN   False     NaN  False   False   \n",
       "761091                                   NaN   False     NaN  False   False   \n",
       "5086863                                  NaN   False     NaN  False   False   \n",
       "5422919                                  NaN   False     NaN  False   False   \n",
       "291159                                   NaN   False     NaN  False   False   \n",
       "6257428                                  0.0   False     0.0  False   False   \n",
       "5877618                                  NaN   False     NaN  False   False   \n",
       "708892                                   NaN   False     NaN  False   False   \n",
       "536617                                   NaN   False     NaN  False   False   \n",
       "5637985                                  0.0   False     0.0  False   False   \n",
       "5690001                                  NaN   False     NaN  False   False   \n",
       "383654                                   0.0   False     0.0  False   False   \n",
       "6027277                                  NaN   False     NaN  False   False   \n",
       "406194                                   NaN   False     NaN  False   False   \n",
       "565755                                   NaN   False     NaN  False   False   \n",
       "811889                                   NaN   False     NaN  False   False   \n",
       "6179046                                  NaN   False     NaN  False   False   \n",
       "5033672                                  NaN   False     NaN  False   False   \n",
       "5341367                                  NaN   False     NaN  False   False   \n",
       "5415652                                  NaN   False     NaN  False   False   \n",
       "5074917                                  NaN   False     NaN  False   False   \n",
       "5808050                                  NaN   False     NaN  False   False   \n",
       "5040012                                  NaN   False     NaN  False   False   \n",
       "1063388                                  NaN   False     NaN  False   False   \n",
       "5677817                                  NaN   False     NaN  False   False   \n",
       "313335                                   NaN   False     NaN  False   False   \n",
       "5882124                                  NaN   False     NaN  False   False   \n",
       "5312539                                  0.0   False     0.0  False   False   \n",
       "872483                                   NaN   False     NaN  False   False   \n",
       "549670                                   NaN   False     NaN  False   False   \n",
       "675443                                   NaN   False     NaN  False   False   \n",
       "5458446                                  NaN   False     NaN  False   False   \n",
       "5396549                                  NaN   False     NaN  False   False   \n",
       "357500                                   NaN   False     NaN  False   False   \n",
       "973919                                   NaN   False     NaN  False   False   \n",
       "6093418                                  NaN   False     NaN  False   False   \n",
       "610989                                   NaN   False     NaN  False   False   \n",
       "5706538                                  NaN   False     NaN  False   False   \n",
       "5466950                                  NaN   False     NaN  False   False   \n",
       "693656                                   NaN   False     NaN  False   False   \n",
       "6322637                                  NaN   False     NaN  False   False   \n",
       "5772784                                  NaN   False     NaN  False   False   \n",
       "600543                                   NaN   False     NaN  False   False   \n",
       "5134770                                  NaN   False     NaN  False   False   \n",
       "5525762                                  NaN   False     NaN  False   False   \n",
       "5905439                                  NaN   False     NaN  False   False   \n",
       "5512374                                  0.0   False     0.0  False   False   \n",
       "521788                                   NaN   False     NaN  False   False   \n",
       "775130                                   NaN   False     NaN  False   False   \n",
       "909742                                   0.0   False     0.0  False   False   \n",
       "5041844                                  NaN   False     NaN  False   False   \n",
       "6053985                                  0.0   False     0.0  False   False   \n",
       "653203                                   NaN   False     NaN  False   False   \n",
       "5768054                                  NaN   False     NaN  False   False   \n",
       "541868                                   NaN   False     NaN  False   False   \n",
       "5309075                                  NaN   False     NaN  False   False   \n",
       "5208691                                  NaN   False     NaN  False   False   \n",
       "5699587                                  NaN   False     NaN  False   False   \n",
       "558939                                   NaN   False     NaN  False   False   \n",
       "5143584                                  NaN   False     NaN  False   False   \n",
       "5165981                                  0.0   False     0.0  False   False   \n",
       "566673                                   NaN   False     NaN  False   False   \n",
       "429233                                   NaN   False     NaN  False   False   \n",
       "5929020                                  NaN   False     NaN  False   False   \n",
       "5207990                                  NaN   False     NaN  False   False   \n",
       "6041871                                  NaN   False     NaN  False   False   \n",
       "5905026                                  NaN   False     NaN  False   False   \n",
       "5464111                                  NaN   False     NaN  False   False   \n",
       "5779787                                  NaN   False     NaN  False   False   \n",
       "767492                                   NaN   False     NaN  False   False   \n",
       "5756759                                  NaN   False     NaN  False   False   \n",
       "5689450                                  NaN   False     NaN  False   False   \n",
       "1065839                                  NaN   False     NaN  False   False   \n",
       "907107                                   0.0   False     0.0  False   False   \n",
       "509921                                   NaN   False     NaN  False   False   \n",
       "676325                                   NaN   False     NaN  False   False   \n",
       "517741                                   NaN   False     NaN  False   False   \n",
       "1007954                                  NaN   False     NaN  False   False   \n",
       "5528968                                  0.0   False     0.0  False   False   \n",
       "5247193                                  NaN   False     NaN  False   False   \n",
       "5129351                                  NaN   False     NaN  False   False   \n",
       "5947689                                  NaN   False     NaN  False   False   \n",
       "927871                                   NaN   False     NaN  False   False   \n",
       "5967514                                  NaN   False     NaN  False   False   \n",
       "5323359                                  NaN   False     NaN  False   False   \n",
       "6044052                                  NaN   False     NaN  False   False   \n",
       "852392                                   NaN   False     NaN  False   False   \n",
       "6172470                                  NaN   False     NaN  False   False   \n",
       "6227710                                  NaN   False     NaN  False   False   \n",
       "\n",
       "         other_disability  other_gender  other_race_or_ethnicity  \\\n",
       "id                                                                 \n",
       "5444896               NaN           NaN                      NaN   \n",
       "390222                NaN           NaN                      NaN   \n",
       "555273                0.0           0.0                      0.0   \n",
       "5167560               NaN           NaN                      NaN   \n",
       "1825271               NaN           NaN                      NaN   \n",
       "595083                NaN           NaN                      NaN   \n",
       "5167671               NaN           NaN                      NaN   \n",
       "716706                NaN           NaN                      NaN   \n",
       "6216305               NaN           NaN                      NaN   \n",
       "5347758               NaN           NaN                      NaN   \n",
       "725142                NaN           NaN                      NaN   \n",
       "5827753               NaN           NaN                      NaN   \n",
       "761091                NaN           NaN                      NaN   \n",
       "5086863               NaN           NaN                      NaN   \n",
       "5422919               NaN           NaN                      NaN   \n",
       "291159                NaN           NaN                      NaN   \n",
       "6257428               0.0           0.0                      0.0   \n",
       "5877618               NaN           NaN                      NaN   \n",
       "708892                NaN           NaN                      NaN   \n",
       "536617                NaN           NaN                      NaN   \n",
       "5637985               0.0           0.0                      0.0   \n",
       "5690001               NaN           NaN                      NaN   \n",
       "383654                0.0           0.0                      0.0   \n",
       "6027277               NaN           NaN                      NaN   \n",
       "406194                NaN           NaN                      NaN   \n",
       "565755                NaN           NaN                      NaN   \n",
       "811889                NaN           NaN                      NaN   \n",
       "6179046               NaN           NaN                      NaN   \n",
       "5033672               NaN           NaN                      NaN   \n",
       "5341367               NaN           NaN                      NaN   \n",
       "5415652               NaN           NaN                      NaN   \n",
       "5074917               NaN           NaN                      NaN   \n",
       "5808050               NaN           NaN                      NaN   \n",
       "5040012               NaN           NaN                      NaN   \n",
       "1063388               NaN           NaN                      NaN   \n",
       "5677817               NaN           NaN                      NaN   \n",
       "313335                NaN           NaN                      NaN   \n",
       "5882124               NaN           NaN                      NaN   \n",
       "5312539               0.0           0.0                      0.0   \n",
       "872483                NaN           NaN                      NaN   \n",
       "549670                NaN           NaN                      NaN   \n",
       "675443                NaN           NaN                      NaN   \n",
       "5458446               NaN           NaN                      NaN   \n",
       "5396549               NaN           NaN                      NaN   \n",
       "357500                NaN           NaN                      NaN   \n",
       "973919                NaN           NaN                      NaN   \n",
       "6093418               NaN           NaN                      NaN   \n",
       "610989                NaN           NaN                      NaN   \n",
       "5706538               NaN           NaN                      NaN   \n",
       "5466950               NaN           NaN                      NaN   \n",
       "693656                NaN           NaN                      NaN   \n",
       "6322637               NaN           NaN                      NaN   \n",
       "5772784               NaN           NaN                      NaN   \n",
       "600543                NaN           NaN                      NaN   \n",
       "5134770               NaN           NaN                      NaN   \n",
       "5525762               NaN           NaN                      NaN   \n",
       "5905439               NaN           NaN                      NaN   \n",
       "5512374               0.0           0.0                      0.0   \n",
       "521788                NaN           NaN                      NaN   \n",
       "775130                NaN           NaN                      NaN   \n",
       "909742                0.0           0.0                      0.0   \n",
       "5041844               NaN           NaN                      NaN   \n",
       "6053985               0.0           0.0                      0.0   \n",
       "653203                NaN           NaN                      NaN   \n",
       "5768054               NaN           NaN                      NaN   \n",
       "541868                NaN           NaN                      NaN   \n",
       "5309075               NaN           NaN                      NaN   \n",
       "5208691               NaN           NaN                      NaN   \n",
       "5699587               NaN           NaN                      NaN   \n",
       "558939                NaN           NaN                      NaN   \n",
       "5143584               NaN           NaN                      NaN   \n",
       "5165981               0.0           0.0                      0.0   \n",
       "566673                NaN           NaN                      NaN   \n",
       "429233                NaN           NaN                      NaN   \n",
       "5929020               NaN           NaN                      NaN   \n",
       "5207990               NaN           NaN                      NaN   \n",
       "6041871               NaN           NaN                      NaN   \n",
       "5905026               NaN           NaN                      NaN   \n",
       "5464111               NaN           NaN                      NaN   \n",
       "5779787               NaN           NaN                      NaN   \n",
       "767492                NaN           NaN                      NaN   \n",
       "5756759               NaN           NaN                      NaN   \n",
       "5689450               NaN           NaN                      NaN   \n",
       "1065839               NaN           NaN                      NaN   \n",
       "907107                0.0           0.0                      0.0   \n",
       "509921                NaN           NaN                      NaN   \n",
       "676325                NaN           NaN                      NaN   \n",
       "517741                NaN           NaN                      NaN   \n",
       "1007954               NaN           NaN                      NaN   \n",
       "5528968               0.0           0.0                      0.0   \n",
       "5247193               NaN           NaN                      NaN   \n",
       "5129351               NaN           NaN                      NaN   \n",
       "5947689               NaN           NaN                      NaN   \n",
       "927871                NaN           NaN                      NaN   \n",
       "5967514               NaN           NaN                      NaN   \n",
       "5323359               NaN           NaN                      NaN   \n",
       "6044052               NaN           NaN                      NaN   \n",
       "852392                NaN           NaN                      NaN   \n",
       "6172470               NaN           NaN                      NaN   \n",
       "6227710               NaN           NaN                      NaN   \n",
       "\n",
       "         other_religion  other_sexual_orientation  physical_disability  \\\n",
       "id                                                                       \n",
       "5444896             NaN                       NaN                  NaN   \n",
       "390222              NaN                       NaN                  NaN   \n",
       "555273              0.0                       0.0                  0.0   \n",
       "5167560             NaN                       NaN                  NaN   \n",
       "1825271             NaN                       NaN                  NaN   \n",
       "595083              NaN                       NaN                  NaN   \n",
       "5167671             NaN                       NaN                  NaN   \n",
       "716706              NaN                       NaN                  NaN   \n",
       "6216305             NaN                       NaN                  NaN   \n",
       "5347758             NaN                       NaN                  NaN   \n",
       "725142              NaN                       NaN                  NaN   \n",
       "5827753             NaN                       NaN                  NaN   \n",
       "761091              NaN                       NaN                  NaN   \n",
       "5086863             NaN                       NaN                  NaN   \n",
       "5422919             NaN                       NaN                  NaN   \n",
       "291159              NaN                       NaN                  NaN   \n",
       "6257428             0.0                       0.0                  0.0   \n",
       "5877618             NaN                       NaN                  NaN   \n",
       "708892              NaN                       NaN                  NaN   \n",
       "536617              NaN                       NaN                  NaN   \n",
       "5637985             0.0                       0.0                  0.0   \n",
       "5690001             NaN                       NaN                  NaN   \n",
       "383654              0.0                       0.0                  0.0   \n",
       "6027277             NaN                       NaN                  NaN   \n",
       "406194              NaN                       NaN                  NaN   \n",
       "565755              NaN                       NaN                  NaN   \n",
       "811889              NaN                       NaN                  NaN   \n",
       "6179046             NaN                       NaN                  NaN   \n",
       "5033672             NaN                       NaN                  NaN   \n",
       "5341367             NaN                       NaN                  NaN   \n",
       "5415652             NaN                       NaN                  NaN   \n",
       "5074917             NaN                       NaN                  NaN   \n",
       "5808050             NaN                       NaN                  NaN   \n",
       "5040012             NaN                       NaN                  NaN   \n",
       "1063388             NaN                       NaN                  NaN   \n",
       "5677817             NaN                       NaN                  NaN   \n",
       "313335              NaN                       NaN                  NaN   \n",
       "5882124             NaN                       NaN                  NaN   \n",
       "5312539             0.0                       0.0                  0.0   \n",
       "872483              NaN                       NaN                  NaN   \n",
       "549670              NaN                       NaN                  NaN   \n",
       "675443              NaN                       NaN                  NaN   \n",
       "5458446             NaN                       NaN                  NaN   \n",
       "5396549             NaN                       NaN                  NaN   \n",
       "357500              NaN                       NaN                  NaN   \n",
       "973919              NaN                       NaN                  NaN   \n",
       "6093418             NaN                       NaN                  NaN   \n",
       "610989              NaN                       NaN                  NaN   \n",
       "5706538             NaN                       NaN                  NaN   \n",
       "5466950             NaN                       NaN                  NaN   \n",
       "693656              NaN                       NaN                  NaN   \n",
       "6322637             NaN                       NaN                  NaN   \n",
       "5772784             NaN                       NaN                  NaN   \n",
       "600543              NaN                       NaN                  NaN   \n",
       "5134770             NaN                       NaN                  NaN   \n",
       "5525762             NaN                       NaN                  NaN   \n",
       "5905439             NaN                       NaN                  NaN   \n",
       "5512374             0.0                       0.0                  0.0   \n",
       "521788              NaN                       NaN                  NaN   \n",
       "775130              NaN                       NaN                  NaN   \n",
       "909742              0.0                       0.0                  0.0   \n",
       "5041844             NaN                       NaN                  NaN   \n",
       "6053985             0.0                       0.0                  0.0   \n",
       "653203              NaN                       NaN                  NaN   \n",
       "5768054             NaN                       NaN                  NaN   \n",
       "541868              NaN                       NaN                  NaN   \n",
       "5309075             NaN                       NaN                  NaN   \n",
       "5208691             NaN                       NaN                  NaN   \n",
       "5699587             NaN                       NaN                  NaN   \n",
       "558939              NaN                       NaN                  NaN   \n",
       "5143584             NaN                       NaN                  NaN   \n",
       "5165981             0.0                       0.0                  0.0   \n",
       "566673              NaN                       NaN                  NaN   \n",
       "429233              NaN                       NaN                  NaN   \n",
       "5929020             NaN                       NaN                  NaN   \n",
       "5207990             NaN                       NaN                  NaN   \n",
       "6041871             NaN                       NaN                  NaN   \n",
       "5905026             NaN                       NaN                  NaN   \n",
       "5464111             NaN                       NaN                  NaN   \n",
       "5779787             NaN                       NaN                  NaN   \n",
       "767492              NaN                       NaN                  NaN   \n",
       "5756759             NaN                       NaN                  NaN   \n",
       "5689450             NaN                       NaN                  NaN   \n",
       "1065839             NaN                       NaN                  NaN   \n",
       "907107              0.0                       0.0                  0.0   \n",
       "509921              NaN                       NaN                  NaN   \n",
       "676325              NaN                       NaN                  NaN   \n",
       "517741              NaN                       NaN                  NaN   \n",
       "1007954             NaN                       NaN                  NaN   \n",
       "5528968             0.0                       0.0                  0.0   \n",
       "5247193             NaN                       NaN                  NaN   \n",
       "5129351             NaN                       NaN                  NaN   \n",
       "5947689             NaN                       NaN                  NaN   \n",
       "927871              NaN                       NaN                  NaN   \n",
       "5967514             NaN                       NaN                  NaN   \n",
       "5323359             NaN                       NaN                  NaN   \n",
       "6044052             NaN                       NaN                  NaN   \n",
       "852392              NaN                       NaN                  NaN   \n",
       "6172470             NaN                       NaN                  NaN   \n",
       "6227710             NaN                       NaN                  NaN   \n",
       "\n",
       "         psychiatric_or_mental_illness  transgender  white  \\\n",
       "id                                                           \n",
       "5444896                          False          NaN  False   \n",
       "390222                           False          NaN  False   \n",
       "555273                           False          0.0  False   \n",
       "5167560                          False          NaN  False   \n",
       "1825271                          False          NaN  False   \n",
       "595083                           False          NaN  False   \n",
       "5167671                          False          NaN  False   \n",
       "716706                           False          NaN  False   \n",
       "6216305                          False          NaN  False   \n",
       "5347758                          False          NaN  False   \n",
       "725142                           False          NaN  False   \n",
       "5827753                          False          NaN  False   \n",
       "761091                           False          NaN  False   \n",
       "5086863                          False          NaN  False   \n",
       "5422919                          False          NaN  False   \n",
       "291159                           False          NaN  False   \n",
       "6257428                          False          0.0  False   \n",
       "5877618                          False          NaN  False   \n",
       "708892                           False          NaN  False   \n",
       "536617                           False          NaN  False   \n",
       "5637985                          False          0.0  False   \n",
       "5690001                          False          NaN  False   \n",
       "383654                           False          0.0  False   \n",
       "6027277                          False          NaN  False   \n",
       "406194                           False          NaN  False   \n",
       "565755                           False          NaN  False   \n",
       "811889                           False          NaN  False   \n",
       "6179046                          False          NaN  False   \n",
       "5033672                          False          NaN  False   \n",
       "5341367                          False          NaN  False   \n",
       "5415652                          False          NaN  False   \n",
       "5074917                          False          NaN  False   \n",
       "5808050                          False          NaN  False   \n",
       "5040012                          False          NaN  False   \n",
       "1063388                          False          NaN  False   \n",
       "5677817                          False          NaN  False   \n",
       "313335                           False          NaN  False   \n",
       "5882124                          False          NaN  False   \n",
       "5312539                          False          0.0  False   \n",
       "872483                           False          NaN  False   \n",
       "549670                           False          NaN  False   \n",
       "675443                           False          NaN  False   \n",
       "5458446                          False          NaN  False   \n",
       "5396549                          False          NaN  False   \n",
       "357500                           False          NaN  False   \n",
       "973919                           False          NaN  False   \n",
       "6093418                          False          NaN  False   \n",
       "610989                           False          NaN  False   \n",
       "5706538                          False          NaN  False   \n",
       "5466950                          False          NaN  False   \n",
       "693656                           False          NaN  False   \n",
       "6322637                          False          NaN  False   \n",
       "5772784                          False          NaN  False   \n",
       "600543                           False          NaN  False   \n",
       "5134770                          False          NaN  False   \n",
       "5525762                          False          NaN  False   \n",
       "5905439                          False          NaN  False   \n",
       "5512374                          False          0.0  False   \n",
       "521788                           False          NaN  False   \n",
       "775130                           False          NaN  False   \n",
       "909742                           False          0.0  False   \n",
       "5041844                          False          NaN  False   \n",
       "6053985                          False          0.0  False   \n",
       "653203                           False          NaN  False   \n",
       "5768054                          False          NaN  False   \n",
       "541868                           False          NaN  False   \n",
       "5309075                          False          NaN  False   \n",
       "5208691                          False          NaN  False   \n",
       "5699587                          False          NaN  False   \n",
       "558939                           False          NaN  False   \n",
       "5143584                          False          NaN  False   \n",
       "5165981                          False          0.0  False   \n",
       "566673                           False          NaN  False   \n",
       "429233                           False          NaN  False   \n",
       "5929020                          False          NaN  False   \n",
       "5207990                          False          NaN  False   \n",
       "6041871                          False          NaN  False   \n",
       "5905026                          False          NaN  False   \n",
       "5464111                          False          NaN  False   \n",
       "5779787                          False          NaN  False   \n",
       "767492                           False          NaN  False   \n",
       "5756759                          False          NaN  False   \n",
       "5689450                          False          NaN  False   \n",
       "1065839                          False          NaN  False   \n",
       "907107                           False          0.0  False   \n",
       "509921                           False          NaN  False   \n",
       "676325                           False          NaN  False   \n",
       "517741                           False          NaN  False   \n",
       "1007954                          False          NaN  False   \n",
       "5528968                          False          0.0  False   \n",
       "5247193                          False          NaN  False   \n",
       "5129351                          False          NaN  False   \n",
       "5947689                          False          NaN  False   \n",
       "927871                           False          NaN  False   \n",
       "5967514                          False          NaN  False   \n",
       "5323359                          False          NaN  False   \n",
       "6044052                          False          NaN  False   \n",
       "852392                           False          NaN  False   \n",
       "6172470                          False          NaN  False   \n",
       "6227710                          False          NaN  False   \n",
       "\n",
       "                          created_date  publication_id  parent_id  article_id  \\\n",
       "id                                                                              \n",
       "5444896  2017-06-20 10:38:11.305896+00              54  5442386.0      345960   \n",
       "390222   2016-07-16 16:48:41.188958+00              21   390062.0      141486   \n",
       "555273   2016-10-26 22:59:27.084162+00              53   555108.0      149566   \n",
       "5167560  2017-04-23 12:34:52.471319+00              54        NaN      328976   \n",
       "1825271  2017-03-06 21:52:16.791716+00              53  1085729.0      317162   \n",
       "595083   2016-11-12 05:19:45.870267+00              22   595008.0      151364   \n",
       "5167671  2017-04-23 13:14:17.691301+00              54        NaN      329025   \n",
       "716706   2016-12-16 04:39:54.372746+00              54   674299.0      155130   \n",
       "6216305  2017-10-24 23:29:28.346830+00              55  6216208.0      392936   \n",
       "5347758  2017-06-02 20:17:29.620335+00              54        NaN      340193   \n",
       "725142   2016-12-18 06:11:24.966813+00              21   724958.0      157225   \n",
       "5827753  2017-08-23 21:31:42.009867+00              54        NaN      369873   \n",
       "761091   2016-12-28 02:13:11.181546+00              54        NaN      158807   \n",
       "5086863  2017-04-03 14:46:48.648776+00              54  5086734.0      324119   \n",
       "5422919  2017-06-15 19:52:00.255718+00              54        NaN      344838   \n",
       "291159   2016-04-21 20:23:02.814628+00              21        NaN       63244   \n",
       "6257428  2017-10-31 12:14:28.900997+00             102        NaN      395334   \n",
       "5877618  2017-09-01 02:51:58.022572+00              55  5876417.0      372881   \n",
       "708892   2016-12-14 16:43:12.928227+00              53   708792.0      156498   \n",
       "536617   2016-10-19 04:12:39.477289+00              21        NaN      148904   \n",
       "5637985  2017-07-22 06:00:36.142877+00              21        NaN      358064   \n",
       "5690001  2017-07-31 15:07:13.856649+00              53  5689946.0      360726   \n",
       "383654   2015-11-17 20:56:30.180896+00              25   383653.0      141164   \n",
       "6027277  2017-09-26 16:42:59.989320+00             102  6027205.0      381959   \n",
       "406194   2016-07-28 23:39:54.828646+00              21        NaN      142541   \n",
       "565755   2016-10-31 20:48:55.432166+00              21   565194.0      149977   \n",
       "811889   2017-01-09 09:29:48.577420+00              21   809924.0      160345   \n",
       "6179046  2017-10-19 14:00:02.507570+00              54  6178961.0      390870   \n",
       "5033672  2017-03-21 15:22:25.098829+00              54        NaN      321032   \n",
       "5341367  2017-06-01 20:11:07.496822+00              13  5340683.0      339490   \n",
       "5415652  2017-06-14 20:23:56.757244+00              53  5409754.0      343874   \n",
       "5074917  2017-03-30 20:45:19.115199+00              53  5074869.0      322741   \n",
       "5808050  2017-08-20 21:22:50.497601+00             102  5806418.0      368647   \n",
       "5040012  2017-03-22 19:44:52.389174+00              54        NaN      321393   \n",
       "1063388  2017-03-01 22:48:03.314604+00              54  1063381.0      316099   \n",
       "5677817  2017-07-28 19:50:51.754377+00              53  5676553.0      360154   \n",
       "313335   2016-05-13 20:26:09.283360+00               6   313228.0       75668   \n",
       "5882124  2017-09-01 21:57:30.507680+00              55  5877936.0      372887   \n",
       "5312539  2017-05-27 11:59:04.576758+00              54  5304935.0      337670   \n",
       "872483   2017-01-21 03:55:32.479786+00              21   872436.0      163011   \n",
       "549670   2016-10-24 23:05:38.896358+00              21   548609.0      149406   \n",
       "675443   2016-12-07 18:59:32.316744+00              53   675104.0      155136   \n",
       "5458446  2017-06-22 14:58:30.945327+00              54        NaN      347350   \n",
       "5396549  2017-06-11 17:20:39.870366+00             102  5396260.0      343102   \n",
       "357500   2016-06-20 05:05:54.672941+00              21   357436.0      139153   \n",
       "973919   2017-02-09 21:14:37.098612+00              54        NaN      167198   \n",
       "6093418  2017-10-06 04:10:33.278544+00              55  6092401.0      385792   \n",
       "610989   2016-11-18 18:19:38.339743+00              53   610955.0      152181   \n",
       "5706538  2017-08-03 03:53:57.612378+00              43  5678200.0      360313   \n",
       "5466950  2017-06-23 19:40:54.981434+00              13  5466941.0      347455   \n",
       "693656   2016-12-11 15:23:35.049818+00              54   684562.0      155499   \n",
       "6322637  2017-11-09 16:37:32.919073+00              54  6321955.0      398850   \n",
       "5772784  2017-08-15 16:47:42.475821+00             102  5771399.0      366454   \n",
       "600543   2016-11-14 20:59:05.669127+00              22        NaN      151608   \n",
       "5134770  2017-04-14 23:29:02.122797+00              53  5134753.0      324719   \n",
       "5525762  2017-07-04 14:49:28.945132+00              54        NaN      351590   \n",
       "5905439  2017-09-06 07:13:57.380740+00              82        NaN      374576   \n",
       "5512374  2017-07-01 20:41:37.504062+00              53  5510353.0      350679   \n",
       "521788   2016-10-11 17:08:29.507636+00              53   521773.0      148116   \n",
       "775130   2016-12-31 20:23:38.078872+00              53   771259.0      159232   \n",
       "909742   2017-01-27 23:01:35.878519+00              54        NaN      164554   \n",
       "5041844  2017-03-23 02:19:55.123133+00              53  5040120.0      321351   \n",
       "6053985  2017-09-30 00:44:15.488260+00              55  6053800.0      383662   \n",
       "653203   2016-12-03 00:44:29.598338+00              21   652922.0      153859   \n",
       "5768054  2017-08-14 21:18:37.365002+00              54  5767387.0      366444   \n",
       "541868   2016-10-21 04:05:28.403972+00              13        NaN      149049   \n",
       "5309075  2017-05-26 17:21:17.680200+00              54        NaN      337957   \n",
       "5208691  2017-05-03 12:18:49.074999+00              54        NaN      331560   \n",
       "5699587  2017-08-01 23:49:47.063278+00              55        NaN      361519   \n",
       "558939   2016-10-28 13:56:14.595471+00              53   558545.0      149770   \n",
       "5143584  2017-04-17 21:23:47.219202+00              54        NaN      327426   \n",
       "5165981  2017-04-22 20:53:36.965769+00              55  5164990.0      328849   \n",
       "566673   2016-11-01 05:02:50.160294+00              53   566591.0      150110   \n",
       "429233   2016-08-13 04:08:32.951058+00              21   428947.0      143578   \n",
       "5929020  2017-09-10 14:45:24.023741+00              54        NaN      376161   \n",
       "5207990  2017-05-03 03:52:41.161333+00              55  5206937.0      331285   \n",
       "6041871  2017-09-28 14:58:31.753988+00              54  6032069.0      382382   \n",
       "5905026  2017-09-06 04:12:13.850119+00              55        NaN      374517   \n",
       "5464111  2017-06-23 12:27:23.965567+00              54        NaN      347728   \n",
       "5779787  2017-08-16 14:54:56.981743+00             102  5779414.0      366993   \n",
       "767492   2016-12-29 21:22:55.803866+00              54   767025.0      159096   \n",
       "5756759  2017-08-12 20:48:42.438363+00              54  5756126.0      365775   \n",
       "5689450  2017-07-31 13:13:32.216426+00              54        NaN      361247   \n",
       "1065839  2017-03-02 15:03:03.813708+00              54  1065632.0      316260   \n",
       "907107   2017-01-27 16:20:14.455600+00              21        NaN      164311   \n",
       "509921   2016-10-06 02:18:31.698833+00              21   508981.0      147620   \n",
       "676325   2016-12-07 21:19:42.647097+00              21   675879.0      155064   \n",
       "517741   2016-10-09 18:55:39.342745+00              53   513419.0      147885   \n",
       "1007954  2017-02-17 16:39:28.241073+00              54  1007851.0      168840   \n",
       "5528968  2017-07-04 23:33:31.491559+00              55        NaN      351704   \n",
       "5247193  2017-05-12 11:35:57.537105+00              54        NaN      334058   \n",
       "5129351  2017-04-13 16:53:32.841972+00              54        NaN      326715   \n",
       "5947689  2017-09-13 19:51:37.548488+00              54        NaN      377417   \n",
       "927871   2017-01-31 18:01:43.343070+00              54   927799.0      165222   \n",
       "5967514  2017-09-17 02:31:48.389817+00             105  5967277.0      378445   \n",
       "5323359  2017-05-29 20:25:16.352274+00              53  5323293.0      336879   \n",
       "6044052  2017-09-28 19:46:22.770697+00             102  6033935.0      382359   \n",
       "852392   2017-01-17 15:27:20.346711+00              53        NaN      162373   \n",
       "6172470  2017-10-18 17:31:34.963777+00             102        NaN      390595   \n",
       "6227710  2017-10-26 16:19:49.616084+00              54  6227476.0      393582   \n",
       "\n",
       "           rating  funny  wow  sad  likes  disagree  sexual_explicit  \\\n",
       "id                                                                     \n",
       "5444896  approved      0    0    0      0         0              0.0   \n",
       "390222   approved      0    0    0      0         0              0.0   \n",
       "555273   approved      0    0    0      1         0              0.0   \n",
       "5167560  approved      0    0    0      0         0              0.0   \n",
       "1825271  approved      0    0    0      2         0              0.0   \n",
       "595083   approved      0    0    0      0         0              0.0   \n",
       "5167671  rejected      0    0    0      0         0              0.0   \n",
       "716706   approved      0    0    0      0         0              0.0   \n",
       "6216305  approved      1    0    0      0         0              0.0   \n",
       "5347758  rejected      0    0    0      0         0              0.0   \n",
       "725142   approved      0    0    0      0         0              0.0   \n",
       "5827753  rejected      0    0    0      0         0              0.0   \n",
       "761091   rejected      0    0    0      0         0              0.0   \n",
       "5086863  approved      0    0    0      0         0              0.0   \n",
       "5422919  rejected      0    0    0      0         0              0.0   \n",
       "291159   approved      0    0    0      0         0              0.0   \n",
       "6257428  approved      0    0    0      0         0              0.0   \n",
       "5877618  approved      0    0    0      0         0              0.0   \n",
       "708892   approved      0    0    0      1         0              0.0   \n",
       "536617   approved      0    0    0      3         0              0.0   \n",
       "5637985  approved      0    0    0      1         0              0.0   \n",
       "5690001  approved      0    0    0      0         0              0.0   \n",
       "383654   approved      0    0    0      0         0              0.0   \n",
       "6027277  approved      0    0    0      0         0              0.0   \n",
       "406194   approved      0    0    0      4         0              0.0   \n",
       "565755   approved      0    0    0      1         0              0.0   \n",
       "811889   approved      0    0    0      0         0              0.0   \n",
       "6179046  approved      0    0    0      7         0              0.0   \n",
       "5033672  rejected      0    0    0      0         0              0.0   \n",
       "5341367  approved      0    0    0      1         0              0.0   \n",
       "5415652  approved      0    0    0      0         0              0.0   \n",
       "5074917  approved      0    0    0      0         0              0.0   \n",
       "5808050  approved      0    0    0      0         0              0.0   \n",
       "5040012  rejected      0    0    0      0         0              0.0   \n",
       "1063388  approved      0    0    0      2         0              0.0   \n",
       "5677817  approved      0    0    0      1         0              0.0   \n",
       "313335   approved      0    0    0      1         0              0.0   \n",
       "5882124  approved      0    0    0      0         0              0.0   \n",
       "5312539  approved      0    0    0      1         0              0.0   \n",
       "872483   approved      1    0    0      2         1              0.0   \n",
       "549670   approved      0    0    0      0         0              0.0   \n",
       "675443   approved      0    0    0      6         0              0.0   \n",
       "5458446  rejected      0    0    0      0         0              0.0   \n",
       "5396549  approved      0    2    0      4         0              0.0   \n",
       "357500   approved      0    0    0      3         0              0.0   \n",
       "973919   approved      0    1    0      0         0              0.0   \n",
       "6093418  approved      0    0    0      0         0              0.0   \n",
       "610989   approved      0    0    0      3         0              0.0   \n",
       "5706538  rejected      0    0    0      0         0              0.0   \n",
       "5466950  approved      0    0    0      0         0              0.0   \n",
       "693656   approved      0    0    0      0         0              0.0   \n",
       "6322637  approved      0    0    0      0         0              0.0   \n",
       "5772784  approved      1    0    0      0         1              0.0   \n",
       "600543   rejected      0    0    0      0         0              0.0   \n",
       "5134770  approved      0    0    0      0         0              0.0   \n",
       "5525762  rejected      0    0    0      0         0              0.0   \n",
       "5905439  approved      0    0    0      0         0              0.0   \n",
       "5512374  approved      0    0    0      2         0              0.0   \n",
       "521788   approved      0    0    0      1         0              0.0   \n",
       "775130   approved      1    0    0      1         0              0.0   \n",
       "909742   approved      1    0    0      0         0              0.0   \n",
       "5041844  approved      0    0    0      1         0              0.0   \n",
       "6053985  approved      0    0    0      0         0              0.0   \n",
       "653203   approved      1    0    0      0         0              0.0   \n",
       "5768054  approved      0    0    0      0         0              0.0   \n",
       "541868   approved      0    0    0      0         0              0.0   \n",
       "5309075  rejected      0    0    0      0         0              0.0   \n",
       "5208691  rejected      0    0    0      0         0              0.0   \n",
       "5699587  approved      0    0    0      0         0              0.0   \n",
       "558939   approved      1    0    0      0         0              0.0   \n",
       "5143584  approved      0    0    0      0         0              0.0   \n",
       "5165981  approved      1    0    0      3         0              0.0   \n",
       "566673   approved      0    0    0      4         0              0.0   \n",
       "429233   approved      0    0    0      0         0              0.0   \n",
       "5929020  rejected      0    0    0      0         0              0.0   \n",
       "5207990  approved      0    0    0      0         0              0.0   \n",
       "6041871  approved      0    0    0      0         1              0.0   \n",
       "5905026  approved      0    0    0      0         0              0.0   \n",
       "5464111  rejected      0    0    0      0         0              0.0   \n",
       "5779787  approved      0    0    0      1         0              0.0   \n",
       "767492   approved      0    0    0      1         0              0.0   \n",
       "5756759  approved      2    1    0      0         0              0.0   \n",
       "5689450  rejected      0    0    0      0         0              0.0   \n",
       "1065839  approved      0    0    0      1         0              0.0   \n",
       "907107   approved      0    0    0      0         0              0.0   \n",
       "509921   approved      0    0    0      0         0              0.0   \n",
       "676325   approved      0    0    0      2         0              0.0   \n",
       "517741   approved      0    0    0      6         0              0.0   \n",
       "1007954  approved      0    0    0      1         0              0.0   \n",
       "5528968  approved      0    0    0      0         0              0.0   \n",
       "5247193  rejected      0    0    0      0         0              0.0   \n",
       "5129351  rejected      0    0    0      0         0              0.0   \n",
       "5947689  rejected      0    0    0      0         0              0.0   \n",
       "927871   approved      0    0    0      1         1              0.0   \n",
       "5967514  approved      0    0    0      0         0              0.0   \n",
       "5323359  approved      0    0    0      1         0              0.0   \n",
       "6044052  approved      0    0    0      0         0              0.0   \n",
       "852392   approved      0    0    0      2         0              0.0   \n",
       "6172470  approved      1    0    0      1         0              0.0   \n",
       "6227710  approved      0    0    0      1         0              0.0   \n",
       "\n",
       "         identity_annotator_count  toxicity_annotator_count  total_length  \\\n",
       "id                                                                          \n",
       "5444896                         0                        10      0.961069   \n",
       "390222                          0                        10      0.928477   \n",
       "555273                          4                         4      0.958315   \n",
       "5167560                         0                         6      0.981872   \n",
       "1825271                         0                        10      0.958315   \n",
       "595083                          0                        10      0.928477   \n",
       "5167671                         0                        10      0.928477   \n",
       "716706                          0                         4      1.000000   \n",
       "6216305                         0                         4      1.000000   \n",
       "5347758                         0                        10      0.928477   \n",
       "725142                          0                        10      0.966988   \n",
       "5827753                         0                        10      0.928477   \n",
       "761091                          0                        10      0.928477   \n",
       "5086863                         0                         5      1.000000   \n",
       "5422919                         0                        10      0.928477   \n",
       "291159                          0                        10      0.958315   \n",
       "6257428                         4                         4      0.957826   \n",
       "5877618                         0                        10      0.963863   \n",
       "708892                          0                        10      0.956730   \n",
       "536617                          0                        10      0.975255   \n",
       "5637985                         4                        10      0.960769   \n",
       "5690001                         0                        10      0.956730   \n",
       "383654                          4                        10      0.953463   \n",
       "6027277                         0                        10      0.956730   \n",
       "406194                          0                        10      0.953463   \n",
       "565755                          0                        10      0.960897   \n",
       "811889                          0                        10      0.928477   \n",
       "6179046                         0                        10      0.928477   \n",
       "5033672                         0                        10      0.928477   \n",
       "5341367                         0                        10      0.928477   \n",
       "5415652                         0                        10      0.960058   \n",
       "5074917                         0                         5      0.953463   \n",
       "5808050                         0                        10      0.953463   \n",
       "5040012                         0                        10      0.928477   \n",
       "1063388                         0                        10      0.960058   \n",
       "5677817                         0                        10      0.961510   \n",
       "313335                          0                        10      0.961069   \n",
       "5882124                         0                        10      0.961510   \n",
       "5312539                         4                         4      0.960769   \n",
       "872483                          0                         4      0.967096   \n",
       "549670                          0                        10      0.928477   \n",
       "675443                          0                        10      0.966988   \n",
       "5458446                         0                        10      0.928477   \n",
       "5396549                         0                        10      1.000000   \n",
       "357500                          0                         4      0.948683   \n",
       "973919                          0                         6      1.000000   \n",
       "6093418                         0                        10      0.957704   \n",
       "610989                          0                         6      0.953463   \n",
       "5706538                         0                        10      0.961524   \n",
       "5466950                         0                        10      0.953463   \n",
       "693656                          0                        10      0.962028   \n",
       "6322637                         0                         4      1.000000   \n",
       "5772784                         0                        10      0.960769   \n",
       "600543                          0                        10      1.000000   \n",
       "5134770                         0                        10      0.956730   \n",
       "5525762                         0                        10      0.928477   \n",
       "5905439                         0                        10      0.960769   \n",
       "5512374                         4                        10      0.974740   \n",
       "521788                          0                        10      0.958315   \n",
       "775130                          0                         6      0.956730   \n",
       "909742                          4                         4      0.948683   \n",
       "5041844                         0                        10      0.958315   \n",
       "6053985                         4                        10      0.928477   \n",
       "653203                          0                        10      0.991750   \n",
       "5768054                         0                        10      0.961880   \n",
       "541868                          0                         4      0.960769   \n",
       "5309075                         0                        10      0.928477   \n",
       "5208691                         0                        10      0.928477   \n",
       "5699587                         0                        10      0.961880   \n",
       "558939                          0                        10      0.960058   \n",
       "5143584                         0                        10      1.000000   \n",
       "5165981                         4                        10      0.953463   \n",
       "566673                          0                        10      0.956730   \n",
       "429233                          0                        10      0.953463   \n",
       "5929020                         0                        10      0.928477   \n",
       "5207990                         0                        10      0.960769   \n",
       "6041871                         0                        10      0.948683   \n",
       "5905026                         0                        10      0.974391   \n",
       "5464111                         0                        10      0.928477   \n",
       "5779787                         0                        10      0.961069   \n",
       "767492                          0                         6      0.983997   \n",
       "5756759                         0                        10      0.960058   \n",
       "5689450                         0                         5      0.928477   \n",
       "1065839                         0                         4      0.963863   \n",
       "907107                          4                         6      0.969912   \n",
       "509921                          0                        10      0.928477   \n",
       "676325                          0                        10      1.000000   \n",
       "517741                          0                        10      0.960058   \n",
       "1007954                         0                        10      0.970725   \n",
       "5528968                         4                         4      0.962159   \n",
       "5247193                         0                        10      0.928477   \n",
       "5129351                         0                        10      0.928477   \n",
       "5947689                         0                        10      0.928477   \n",
       "927871                          0                         4      0.960769   \n",
       "5967514                         0                        10      0.928477   \n",
       "5323359                         0                        10      0.958315   \n",
       "6044052                         0                        10      0.961510   \n",
       "852392                          0                         5      0.958315   \n",
       "6172470                         0                         4      0.958315   \n",
       "6227710                         0                         4      0.958315   \n",
       "\n",
       "         capitals  caps_vs_length  num_exclamation_marks  num_question_marks  \\\n",
       "id                                                                             \n",
       "5444896       0.0             0.0               0.000000            0.192214   \n",
       "390222        0.0             0.0               0.000000            0.185695   \n",
       "555273        0.0             0.0               0.000000            0.000000   \n",
       "5167560       0.0             0.0               0.092051            0.000000   \n",
       "1825271       0.0             0.0               0.000000            0.000000   \n",
       "595083        0.0             0.0               0.000000            0.185695   \n",
       "5167671       0.0             0.0               0.000000            0.000000   \n",
       "716706        0.0             0.0               0.000000            0.000000   \n",
       "6216305       0.0             0.0               0.000000            0.000000   \n",
       "5347758       0.0             0.0               0.000000            0.000000   \n",
       "725142        0.0             0.0               0.000000            0.000000   \n",
       "5827753       0.0             0.0               0.000000            0.000000   \n",
       "761091        0.0             0.0               0.000000            0.000000   \n",
       "5086863       0.0             0.0               0.000000            0.000000   \n",
       "5422919       0.0             0.0               0.000000            0.000000   \n",
       "291159        0.0             0.0               0.000000            0.000000   \n",
       "6257428       0.0             0.0               0.000000            0.000000   \n",
       "5877618       0.0             0.0               0.000000            0.000000   \n",
       "708892        0.0             0.0               0.000000            0.000000   \n",
       "536617        0.0             0.0               0.000000            0.000000   \n",
       "5637985       0.0             0.0               0.000000            0.000000   \n",
       "5690001       0.0             0.0               0.000000            0.191346   \n",
       "383654        0.0             0.0               0.000000            0.000000   \n",
       "6027277       0.0             0.0               0.000000            0.191346   \n",
       "406194        0.0             0.0               0.000000            0.000000   \n",
       "565755        0.0             0.0               0.000000            0.000000   \n",
       "811889        0.0             0.0               0.000000            0.185695   \n",
       "6179046       0.0             0.0               0.000000            0.185695   \n",
       "5033672       0.0             0.0               0.000000            0.000000   \n",
       "5341367       0.0             0.0               0.000000            0.185695   \n",
       "5415652       0.0             0.0               0.000000            0.000000   \n",
       "5074917       0.0             0.0               0.000000            0.000000   \n",
       "5808050       0.0             0.0               0.000000            0.000000   \n",
       "5040012       0.0             0.0               0.000000            0.000000   \n",
       "1063388       0.0             0.0               0.000000            0.192012   \n",
       "5677817       0.0             0.0               0.000000            0.192302   \n",
       "313335        0.0             0.0               0.000000            0.192214   \n",
       "5882124       0.0             0.0               0.000000            0.192302   \n",
       "5312539       0.0             0.0               0.000000            0.000000   \n",
       "872483        0.0             0.0               0.000000            0.000000   \n",
       "549670        0.0             0.0               0.000000            0.185695   \n",
       "675443        0.0             0.0               0.000000            0.000000   \n",
       "5458446       0.0             0.0               0.000000            0.000000   \n",
       "5396549       0.0             0.0               0.000000            0.000000   \n",
       "357500        0.0             0.0               0.000000            0.000000   \n",
       "973919        0.0             0.0               0.000000            0.000000   \n",
       "6093418       0.0             0.0               0.000000            0.000000   \n",
       "610989        0.0             0.0               0.000000            0.000000   \n",
       "5706538       0.0             0.0               0.000000            0.000000   \n",
       "5466950       0.0             0.0               0.000000            0.000000   \n",
       "693656        0.0             0.0               0.000000            0.192406   \n",
       "6322637       0.0             0.0               0.000000            0.000000   \n",
       "5772784       0.0             0.0               0.000000            0.000000   \n",
       "600543        0.0             0.0               0.000000            0.000000   \n",
       "5134770       0.0             0.0               0.000000            0.191346   \n",
       "5525762       0.0             0.0               0.000000            0.000000   \n",
       "5905439       0.0             0.0               0.000000            0.000000   \n",
       "5512374       0.0             0.0               0.000000            0.000000   \n",
       "521788        0.0             0.0               0.000000            0.000000   \n",
       "775130        0.0             0.0               0.000000            0.191346   \n",
       "909742        0.0             0.0               0.000000            0.000000   \n",
       "5041844       0.0             0.0               0.000000            0.000000   \n",
       "6053985       0.0             0.0               0.000000            0.185695   \n",
       "653203        0.0             0.0               0.000000            0.000000   \n",
       "5768054       0.0             0.0               0.000000            0.192376   \n",
       "541868        0.0             0.0               0.000000            0.000000   \n",
       "5309075       0.0             0.0               0.000000            0.000000   \n",
       "5208691       0.0             0.0               0.000000            0.000000   \n",
       "5699587       0.0             0.0               0.000000            0.000000   \n",
       "558939        0.0             0.0               0.000000            0.192012   \n",
       "5143584       0.0             0.0               0.000000            0.000000   \n",
       "5165981       0.0             0.0               0.000000            0.000000   \n",
       "566673        0.0             0.0               0.000000            0.191346   \n",
       "429233        0.0             0.0               0.000000            0.000000   \n",
       "5929020       0.0             0.0               0.000000            0.000000   \n",
       "5207990       0.0             0.0               0.000000            0.000000   \n",
       "6041871       0.0             0.0               0.000000            0.000000   \n",
       "5905026       0.0             0.0               0.000000            0.000000   \n",
       "5464111       0.0             0.0               0.000000            0.000000   \n",
       "5779787       0.0             0.0               0.000000            0.192214   \n",
       "767492        0.0             0.0               0.000000            0.000000   \n",
       "5756759       0.0             0.0               0.000000            0.192012   \n",
       "5689450       0.0             0.0               0.000000            0.000000   \n",
       "1065839       0.0             0.0               0.000000            0.000000   \n",
       "907107        0.0             0.0               0.138559            0.000000   \n",
       "509921        0.0             0.0               0.000000            0.185695   \n",
       "676325        0.0             0.0               0.000000            0.000000   \n",
       "517741        0.0             0.0               0.000000            0.192012   \n",
       "1007954       0.0             0.0               0.000000            0.000000   \n",
       "5528968       0.0             0.0               0.000000            0.000000   \n",
       "5247193       0.0             0.0               0.000000            0.000000   \n",
       "5129351       0.0             0.0               0.000000            0.000000   \n",
       "5947689       0.0             0.0               0.000000            0.000000   \n",
       "927871        0.0             0.0               0.000000            0.000000   \n",
       "5967514       0.0             0.0               0.000000            0.185695   \n",
       "5323359       0.0             0.0               0.000000            0.000000   \n",
       "6044052       0.0             0.0               0.000000            0.192302   \n",
       "852392        0.0             0.0               0.000000            0.000000   \n",
       "6172470       0.0             0.0               0.000000            0.000000   \n",
       "6227710       0.0             0.0               0.000000            0.000000   \n",
       "\n",
       "         num_punctuation  num_symbols  num_words  num_unique_words  \\\n",
       "id                                                                   \n",
       "5444896         0.000000     0.000000   0.192214          0.048053   \n",
       "390222          0.000000     0.000000   0.185695          0.185695   \n",
       "555273          0.063888     0.000000   0.191663          0.191663   \n",
       "5167560         0.000000     0.000000   0.153418          0.061367   \n",
       "1825271         0.063888     0.000000   0.191663          0.191663   \n",
       "595083          0.000000     0.000000   0.185695          0.185695   \n",
       "5167671         0.185695     0.000000   0.185695          0.185695   \n",
       "716706          0.000000     0.000000   0.000000          0.000000   \n",
       "6216305         0.000000     0.000000   0.000000          0.000000   \n",
       "5347758         0.185695     0.000000   0.185695          0.185695   \n",
       "725142          0.080582     0.000000   0.161165          0.161165   \n",
       "5827753         0.185695     0.000000   0.185695          0.185695   \n",
       "761091          0.185695     0.000000   0.185695          0.185695   \n",
       "5086863         0.000000     0.000000   0.000000          0.000000   \n",
       "5422919         0.185695     0.000000   0.185695          0.185695   \n",
       "291159          0.063888     0.000000   0.191663          0.191663   \n",
       "6257428         0.000000     0.000000   0.191565          0.191565   \n",
       "5877618         0.080322     0.080322   0.160644          0.160644   \n",
       "708892          0.095673     0.000000   0.191346          0.191346   \n",
       "536617          0.032509     0.000000   0.195051          0.097526   \n",
       "5637985         0.000000     0.000000   0.160128          0.160128   \n",
       "5690001         0.000000     0.000000   0.191346          0.095673   \n",
       "383654          0.095346     0.000000   0.190693          0.190693   \n",
       "6027277         0.000000     0.000000   0.191346          0.095673   \n",
       "406194          0.095346     0.000000   0.190693          0.190693   \n",
       "565755          0.056523     0.113047   0.169570          0.169570   \n",
       "811889          0.000000     0.000000   0.185695          0.185695   \n",
       "6179046         0.000000     0.000000   0.185695          0.185695   \n",
       "5033672         0.185695     0.000000   0.185695          0.185695   \n",
       "5341367         0.000000     0.000000   0.185695          0.185695   \n",
       "5415652         0.192012     0.000000   0.192012          0.064004   \n",
       "5074917         0.095346     0.000000   0.190693          0.190693   \n",
       "5808050         0.095346     0.000000   0.190693          0.190693   \n",
       "5040012         0.185695     0.000000   0.185695          0.185695   \n",
       "1063388         0.000000     0.000000   0.192012          0.064004   \n",
       "5677817         0.000000     0.000000   0.192302          0.038460   \n",
       "313335          0.000000     0.000000   0.192214          0.048053   \n",
       "5882124         0.000000     0.000000   0.192302          0.038460   \n",
       "5312539         0.000000     0.000000   0.160128          0.160128   \n",
       "872483          0.056888     0.000000   0.170664          0.170664   \n",
       "549670          0.000000     0.000000   0.185695          0.185695   \n",
       "675443          0.080582     0.000000   0.161165          0.161165   \n",
       "5458446         0.185695     0.000000   0.185695          0.185695   \n",
       "5396549         0.000000     0.000000   0.000000          0.000000   \n",
       "357500          0.000000     0.158114   0.158114          0.158114   \n",
       "973919          0.000000     0.000000   0.000000          0.000000   \n",
       "6093418         0.159617     0.000000   0.159617          0.159617   \n",
       "610989          0.095346     0.000000   0.190693          0.190693   \n",
       "5706538         0.137361     0.000000   0.137361          0.137361   \n",
       "5466950         0.095346     0.000000   0.190693          0.190693   \n",
       "693656          0.000000     0.000000   0.192406          0.021378   \n",
       "6322637         0.000000     0.000000   0.000000          0.000000   \n",
       "5772784         0.000000     0.000000   0.160128          0.160128   \n",
       "600543          0.000000     0.000000   0.000000          0.000000   \n",
       "5134770         0.000000     0.000000   0.191346          0.095673   \n",
       "5525762         0.185695     0.000000   0.185695          0.185695   \n",
       "5905439         0.000000     0.000000   0.160128          0.160128   \n",
       "5512374         0.000000     0.000000   0.194948          0.097474   \n",
       "521788          0.063888     0.000000   0.191663          0.191663   \n",
       "775130          0.000000     0.000000   0.191346          0.095673   \n",
       "909742          0.158114     0.000000   0.158114          0.158114   \n",
       "5041844         0.063888     0.000000   0.191663          0.191663   \n",
       "6053985         0.000000     0.000000   0.185695          0.185695   \n",
       "653203          0.000000     0.000000   0.121439          0.040480   \n",
       "5768054         0.000000     0.000000   0.192376          0.027482   \n",
       "541868          0.000000     0.000000   0.160128          0.160128   \n",
       "5309075         0.185695     0.000000   0.185695          0.185695   \n",
       "5208691         0.185695     0.000000   0.185695          0.185695   \n",
       "5699587         0.000000     0.192376   0.192376          0.027482   \n",
       "558939          0.000000     0.000000   0.192012          0.064004   \n",
       "5143584         0.000000     0.000000   0.000000          0.000000   \n",
       "5165981         0.095346     0.000000   0.190693          0.190693   \n",
       "566673          0.000000     0.000000   0.191346          0.095673   \n",
       "429233          0.095346     0.000000   0.190693          0.190693   \n",
       "5929020         0.185695     0.000000   0.185695          0.185695   \n",
       "5207990         0.000000     0.000000   0.160128          0.160128   \n",
       "6041871         0.000000     0.158114   0.158114          0.158114   \n",
       "5905026         0.000000     0.000000   0.149906          0.149906   \n",
       "5464111         0.185695     0.000000   0.185695          0.185695   \n",
       "5779787         0.000000     0.000000   0.192214          0.048053   \n",
       "767492          0.027718     0.027718   0.152450          0.083155   \n",
       "5756759         0.000000     0.000000   0.192012          0.064004   \n",
       "5689450         0.185695     0.000000   0.185695          0.185695   \n",
       "1065839         0.080322     0.080322   0.160644          0.160644   \n",
       "907107          0.023093     0.023093   0.184745          0.069279   \n",
       "509921          0.000000     0.000000   0.185695          0.185695   \n",
       "676325          0.000000     0.000000   0.000000          0.000000   \n",
       "517741          0.000000     0.000000   0.192012          0.064004   \n",
       "1007954         0.000000     0.000000   0.138675          0.138675   \n",
       "5528968         0.000000     0.192432   0.192432          0.013745   \n",
       "5247193         0.185695     0.000000   0.185695          0.185695   \n",
       "5129351         0.185695     0.000000   0.185695          0.185695   \n",
       "5947689         0.185695     0.000000   0.185695          0.185695   \n",
       "927871          0.000000     0.000000   0.160128          0.160128   \n",
       "5967514         0.000000     0.000000   0.185695          0.185695   \n",
       "5323359         0.063888     0.000000   0.191663          0.191663   \n",
       "6044052         0.000000     0.000000   0.192302          0.038460   \n",
       "852392          0.063888     0.000000   0.191663          0.191663   \n",
       "6172470         0.063888     0.000000   0.191663          0.191663   \n",
       "6227710         0.063888     0.000000   0.191663          0.191663   \n",
       "\n",
       "         words_vs_unique  num_smilies  length  \n",
       "id                                             \n",
       "5444896         0.012013          0.0       0  \n",
       "390222          0.185695          0.0       0  \n",
       "555273          0.063888          0.0       0  \n",
       "5167560         0.012273          0.0       0  \n",
       "1825271         0.063888          0.0       0  \n",
       "595083          0.185695          0.0       0  \n",
       "5167671         0.185695          0.0       0  \n",
       "716706          0.000000          0.0       0  \n",
       "6216305         0.000000          0.0       0  \n",
       "5347758         0.185695          0.0       0  \n",
       "725142          0.080582          0.0       0  \n",
       "5827753         0.185695          0.0       0  \n",
       "761091          0.185695          0.0       0  \n",
       "5086863         0.000000          0.0       0  \n",
       "5422919         0.185695          0.0       0  \n",
       "291159          0.063888          0.0       0  \n",
       "6257428         0.095783          0.0       0  \n",
       "5877618         0.080322          0.0       0  \n",
       "708892          0.047836          0.0       0  \n",
       "536617          0.016254          0.0       0  \n",
       "5637985         0.160128          0.0       0  \n",
       "5690001         0.047836          0.0       0  \n",
       "383654          0.095346          0.0       0  \n",
       "6027277         0.047836          0.0       0  \n",
       "406194          0.095346          0.0       0  \n",
       "565755          0.056523          0.0       0  \n",
       "811889          0.185695          0.0       0  \n",
       "6179046         0.185695          0.0       0  \n",
       "5033672         0.185695          0.0       0  \n",
       "5341367         0.185695          0.0       0  \n",
       "5415652         0.021335          0.0       0  \n",
       "5074917         0.095346          0.0       0  \n",
       "5808050         0.095346          0.0       0  \n",
       "5040012         0.185695          0.0       0  \n",
       "1063388         0.021335          0.0       0  \n",
       "5677817         0.007692          0.0       0  \n",
       "313335          0.012013          0.0       0  \n",
       "5882124         0.007692          0.0       0  \n",
       "5312539         0.160128          0.0       0  \n",
       "872483          0.056888          0.0       0  \n",
       "549670          0.185695          0.0       0  \n",
       "675443          0.080582          0.0       0  \n",
       "5458446         0.185695          0.0       0  \n",
       "5396549         0.000000          0.0       0  \n",
       "357500          0.158114          0.0       0  \n",
       "973919          0.000000          0.0       0  \n",
       "6093418         0.079809          0.0       0  \n",
       "610989          0.095346          0.0       0  \n",
       "5706538         0.137361          0.0       0  \n",
       "5466950         0.095346          0.0       0  \n",
       "693656          0.002375          0.0       0  \n",
       "6322637         0.000000          0.0       0  \n",
       "5772784         0.160128          0.0       0  \n",
       "600543          0.000000          0.0       0  \n",
       "5134770         0.047836          0.0       0  \n",
       "5525762         0.185695          0.0       0  \n",
       "5905439         0.160128          0.0       0  \n",
       "5512374         0.048737          0.0       0  \n",
       "521788          0.063888          0.0       0  \n",
       "775130          0.047836          0.0       0  \n",
       "909742          0.158114          0.0       0  \n",
       "5041844         0.063888          0.0       0  \n",
       "6053985         0.185695          0.0       0  \n",
       "653203          0.006747          0.0       0  \n",
       "5768054         0.003926          0.0       0  \n",
       "541868          0.160128          0.0       0  \n",
       "5309075         0.185695          0.0       0  \n",
       "5208691         0.185695          0.0       0  \n",
       "5699587         0.003926          0.0       0  \n",
       "558939          0.021335          0.0       0  \n",
       "5143584         0.000000          0.0       0  \n",
       "5165981         0.095346          0.0       0  \n",
       "566673          0.047836          0.0       0  \n",
       "429233          0.095346          0.0       0  \n",
       "5929020         0.185695          0.0       0  \n",
       "5207990         0.160128          0.0       0  \n",
       "6041871         0.158114          0.0       0  \n",
       "5905026         0.074953          0.0       0  \n",
       "5464111         0.185695          0.0       0  \n",
       "5779787         0.012013          0.0       0  \n",
       "767492          0.007560          0.0       0  \n",
       "5756759         0.021335          0.0       0  \n",
       "5689450         0.185695          0.0       0  \n",
       "1065839         0.080322          0.0       0  \n",
       "907107          0.008660          0.0       0  \n",
       "509921          0.185695          0.0       0  \n",
       "676325          0.000000          0.0       0  \n",
       "517741          0.021335          0.0       0  \n",
       "1007954         0.138675          0.0       0  \n",
       "5528968         0.000982          0.0       0  \n",
       "5247193         0.185695          0.0       0  \n",
       "5129351         0.185695          0.0       0  \n",
       "5947689         0.185695          0.0       0  \n",
       "927871          0.160128          0.0       0  \n",
       "5967514         0.185695          0.0       0  \n",
       "5323359         0.063888          0.0       0  \n",
       "6044052         0.007692          0.0       0  \n",
       "852392          0.063888          0.0       0  \n",
       "6172470         0.063888          0.0       0  \n",
       "6227710         0.063888          0.0       0  "
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sorted_train.head(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:31.144787Z",
     "start_time": "2019-06-01T04:12:30.763808Z"
    }
   },
   "outputs": [],
   "source": [
    "# Overall\n",
    "weights = np.ones((len(train),)) / 4\n",
    "# Subgroup\n",
    "weights += (train[identity_columns].fillna(0).values>=0.5).sum(axis=1).astype(bool).astype(np.int) / 4\n",
    "# Background Positive, Subgroup Negative\n",
    "weights += (( (train['target'].values>=0.5).astype(bool).astype(np.int) +\n",
    "   (train[identity_columns].fillna(0).values<0.5).sum(axis=1).astype(bool).astype(np.int) ) > 1 ).astype(bool).astype(np.int) / 4\n",
    "# Background Negative, Subgroup Positive\n",
    "weights += (( (train['target'].values<0.5).astype(bool).astype(np.int) +\n",
    "   (train[identity_columns].fillna(0).values>=0.5).sum(axis=1).astype(bool).astype(np.int) ) > 1 ).astype(bool).astype(np.int) / 4\n",
    "loss_weight = 1.0 / weights.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:31.150771Z",
     "start_time": "2019-06-01T04:12:31.145785Z"
    }
   },
   "outputs": [],
   "source": [
    "def custom_loss(data, targets):\n",
    "    ''' Define custom loss function for weighted BCE on 'target' column '''\n",
    "    bce_loss_1 = nn.BCEWithLogitsLoss(weight=targets[:,1:2])(data[:,:1],targets[:,:1])\n",
    "    bce_loss_2 = nn.BCEWithLogitsLoss()(data[:,1:],targets[:,2:])\n",
    "    return (bce_loss_1 * loss_weight) + bce_loss_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Binning Different Length Sentences as A list of DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:31.161743Z",
     "start_time": "2019-06-01T04:12:31.151769Z"
    }
   },
   "outputs": [],
   "source": [
    "boundary_list = [0,300000,700000, 1000000,1200000,1350000,1500000, len(train)-1]\n",
    "sorted_train_list = []\n",
    "for i in range(len(boundary_list)):\n",
    "    if i<len(boundary_list)-1: # except the last one\n",
    "        temp_train = sorted_train[boundary_list[i]:boundary_list[i+1]]\n",
    "        sorted_train_list.append(temp_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stat Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:31.270452Z",
     "start_time": "2019-06-01T04:12:31.163738Z"
    }
   },
   "outputs": [],
   "source": [
    "sorted_train_stat_list = []\n",
    "for sorted_train in sorted_train_list:\n",
    "    temp_train_stat = sorted_train[statistic_columns].values\n",
    "    sorted_train_stat_list.append(temp_train_stat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:12:31.295386Z",
     "start_time": "2019-06-01T04:12:31.271450Z"
    }
   },
   "outputs": [],
   "source": [
    "test_stat = test[statistic_columns].values\n",
    "valid_stat = valid[statistic_columns].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## tokenize the sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:25.986259Z",
     "start_time": "2019-06-01T04:12:31.297381Z"
    }
   },
   "outputs": [],
   "source": [
    "sorted_train_x_list = [] # a list of train_x\n",
    "for sorted_train in sorted_train_list:\n",
    "    temp_train_x = tokenizer.texts_to_sequences(sorted_train['comment_text'])\n",
    "    sorted_train_x_list.append(temp_train_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:32.592601Z",
     "start_time": "2019-06-01T04:13:25.990193Z"
    }
   },
   "outputs": [],
   "source": [
    "test_x = tokenizer.texts_to_sequences(test['comment_text'])\n",
    "valid_x = tokenizer.texts_to_sequences(valid['comment_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Padding & Set Y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:42.428289Z",
     "start_time": "2019-06-01T04:13:32.594577Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[13, 28, 46, 65, 86, 122, 220]\n"
     ]
    }
   ],
   "source": [
    "padded_train_x_list = []\n",
    "PD_size_list = []\n",
    "for sorted_train_x in sorted_train_x_list:\n",
    "    PD_size = min(max(len(s)for s in sorted_train_x),maxlen) ## 取subset里最长 且不超过maxlen\n",
    "    PD_size_list.append(PD_size)\n",
    "    temp_padded_train_x = pad_sequences(sorted_train_x, maxlen=PD_size)\n",
    "    padded_train_x_list.append(temp_padded_train_x)\n",
    "print(PD_size_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.720838Z",
     "start_time": "2019-06-01T04:13:42.429251Z"
    }
   },
   "outputs": [],
   "source": [
    "valid_x = pad_sequences(valid_x, maxlen=maxlen)\n",
    "test_x = pad_sequences(test_x, maxlen=maxlen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.731770Z",
     "start_time": "2019-06-01T04:13:43.721797Z"
    }
   },
   "outputs": [],
   "source": [
    "valid_y = valid[['target','target', 'severe_toxicity', 'obscene', 'identity_attack', 'insult', 'threat']].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.805572Z",
     "start_time": "2019-06-01T04:13:43.732768Z"
    }
   },
   "outputs": [],
   "source": [
    "sorted_train_y_list = []\n",
    "for sorted_train in sorted_train_list:\n",
    "    temp_train_y = sorted_train[['target','target', 'severe_toxicity', 'obscene', 'identity_attack', 'insult', 'threat']].values\n",
    "    sorted_train_y_list.append(temp_train_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build Up Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.812554Z",
     "start_time": "2019-06-01T04:13:43.807567Z"
    }
   },
   "outputs": [],
   "source": [
    "class SpatialDropout(nn.Dropout2d):\n",
    "    def forward(self, x):\n",
    "        x = x.unsqueeze(2)    # (N, T, 1, K)\n",
    "        x = x.permute(0, 3, 2, 1)  # (N, K, 1, T)\n",
    "        x = super(SpatialDropout, self).forward(x)  # (N, K, 1, T), some features are masked\n",
    "        x = x.permute(0, 3, 2, 1)  # (N, T, 1, K)\n",
    "        x = x.squeeze(2)  # (N, T, K)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.820533Z",
     "start_time": "2019-06-01T04:13:43.814549Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "289642\n"
     ]
    }
   ],
   "source": [
    "max_features = min(len(tokenizer.word_index) + 1,max_features)\n",
    "print(max_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.825521Z",
     "start_time": "2019-06-01T04:13:43.821531Z"
    }
   },
   "outputs": [],
   "source": [
    "num_aux_targets = 6\n",
    "NUM_MODELS = 2\n",
    "LSTM_UNITS = 128\n",
    "DENSE_HIDDEN_UNITS = 4 * LSTM_UNITS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.845468Z",
     "start_time": "2019-06-01T04:13:43.827516Z"
    }
   },
   "outputs": [],
   "source": [
    "class NeuralNet(nn.Module):\n",
    "    def __init__(self, embedding_matrix, num_aux_targets,):\n",
    "        super(NeuralNet, self).__init__()\n",
    "        embed_size = embedding_matrix.shape[1]\n",
    "        \n",
    "        self.embedding = nn.Embedding(max_features, embed_size)\n",
    "        self.embedding.weight = nn.Parameter(torch.tensor(embedding_matrix, dtype=torch.float32))\n",
    "        self.embedding.weight.requires_grad = False\n",
    "        self.embedding_dropout = SpatialDropout(0.3)\n",
    "        \n",
    "        self.lstm1 = nn.LSTM(embed_size, LSTM_UNITS, bidirectional=True, batch_first=True)\n",
    "        self.lstm2 = nn.LSTM(LSTM_UNITS * 2, LSTM_UNITS, bidirectional=True, batch_first=True)\n",
    "    \n",
    "        self.linear1 = nn.Linear(DENSE_HIDDEN_UNITS+64, DENSE_HIDDEN_UNITS+64)\n",
    "        self.linear2 = nn.Linear(DENSE_HIDDEN_UNITS+64, DENSE_HIDDEN_UNITS+64)\n",
    "        \n",
    "        self.linear_stat = nn.Linear(11, 64)\n",
    "        \n",
    "        \n",
    "        self.linear_out = nn.Linear(DENSE_HIDDEN_UNITS+64, 1)\n",
    "        self.linear_aux_out = nn.Linear(DENSE_HIDDEN_UNITS+64, num_aux_targets)\n",
    "        \n",
    "    def forward(self, x,stat_x,lengths=None):\n",
    "        h_embedding = self.embedding(x.long())\n",
    "        h_embedding = self.embedding_dropout(h_embedding)\n",
    "        \n",
    "        h_lstm1, _ = self.lstm1(h_embedding)\n",
    "        h_lstm2, _ = self.lstm2(h_lstm1)\n",
    "        \n",
    "        \n",
    "        # global average pooling\n",
    "        avg_pool = torch.mean(h_lstm2, 1)\n",
    "        # global max pooling\n",
    "        max_pool, _ = torch.max(h_lstm2, 1)\n",
    "        #stat feature out\n",
    "        stat_out = self.linear_stat(stat_x)\n",
    "        \n",
    "        \n",
    "        h_conc = torch.cat((max_pool, avg_pool,stat_out), 1)\n",
    "        h_conc_linear1  = F.relu(self.linear1(h_conc))\n",
    "        h_conc_linear2  = F.relu(self.linear2(h_conc))\n",
    "        \n",
    "        hidden = h_conc + h_conc_linear1 + h_conc_linear2\n",
    "        \n",
    "        result = self.linear_out(hidden)\n",
    "        aux_result = self.linear_aux_out(hidden)\n",
    "        out = torch.cat([result, aux_result], 1)\n",
    "        \n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T18:16:29.430238Z",
     "start_time": "2019-05-31T18:16:29.418270Z"
    }
   },
   "outputs": [],
   "source": [
    "# class NeuralNet(nn.Module):\n",
    "#     def __init__(self, embedding_matrix):\n",
    "#         super(NeuralNet, self).__init__()\n",
    "\n",
    "#         lstm_hidden_size = 120\n",
    "#         gru_hidden_size = 60\n",
    "        \n",
    "#         self.gru_hidden_size = gru_hidden_size\n",
    "#         self.embedding = nn.Embedding(max_features, embed_size)\n",
    "#         self.embedding.weight = nn.Parameter(torch.tensor(embedding_matrix, dtype=torch.float32))\n",
    "#         self.embedding.weight.requires_grad = False\n",
    "#         self.embedding_dropout = nn.Dropout2d(0.25)\n",
    "\n",
    "#         self.lstm = nn.LSTM(embed_size, lstm_hidden_size, bidirectional=True, batch_first=True)\n",
    "#         self.gru = nn.GRU(lstm_hidden_size * 2, gru_hidden_size, bidirectional=True, batch_first=True)\n",
    "\n",
    "#         self.linear = nn.Linear(gru_hidden_size * 6, 32)\n",
    "#         self.relu = nn.ReLU()\n",
    "#         self.dropout = nn.Dropout(0.25)\n",
    "#         self.linear_aux_out = nn.Linear(32, 5)\n",
    "#         self.out = nn.Linear(32, 1)\n",
    "\n",
    "#     def apply_spatial_dropout(self, h_embedding):\n",
    "#         h_embedding = h_embedding.transpose(1, 2).unsqueeze(2)\n",
    "#         h_embedding = self.embedding_dropout(h_embedding).squeeze(2).transpose(1, 2)\n",
    "#         return h_embedding\n",
    "\n",
    "#     def forward(self, x):\n",
    "#         h_embedding = self.embedding(x)\n",
    "#         h_embedding = self.apply_spatial_dropout(h_embedding)\n",
    "\n",
    "#         h_lstm, _ = self.lstm(h_embedding)\n",
    "#         h_gru, hh_gru = self.gru(h_lstm)\n",
    "\n",
    "#         hh_gru = hh_gru.view(-1, self.gru_hidden_size * 2) #把最后一个h展开 N * GRUOUT*2\n",
    "\n",
    "#         avg_pool = torch.mean(h_gru, 1)    # N * GRUOUT*2\n",
    "#         max_pool, _ = torch.max(h_gru, 1)  # N * GRUOUT*2\n",
    "\n",
    "#         conc = torch.cat((hh_gru, avg_pool, max_pool), 1)\n",
    "#         conc = self.relu(self.linear(conc))\n",
    "#         conc = self.dropout(conc)\n",
    "#         aux_result = self.linear_aux_out(conc)\n",
    "#         result = self.out(conc)\n",
    "#         out = torch.cat([result, aux_result], 1)\n",
    "#         return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation Fuctions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.870400Z",
     "start_time": "2019-06-01T04:13:43.847461Z"
    }
   },
   "outputs": [],
   "source": [
    "SUBGROUP_AUC = 'subgroup_auc'\n",
    "BPSN_AUC = 'bpsn_auc'  # stands for background positive, subgroup negative\n",
    "BNSP_AUC = 'bnsp_auc'  # stands for background negative, subgroup positive\n",
    "\n",
    "def compute_auc(y_true, y_pred):\n",
    "    try:\n",
    "        return metrics.roc_auc_score(y_true, y_pred)\n",
    "    except ValueError:\n",
    "        return np.nan\n",
    "\n",
    "def compute_subgroup_auc(df, subgroup, label, model_name):\n",
    "    subgroup_examples = df[df[subgroup]]\n",
    "    return compute_auc(subgroup_examples[label], subgroup_examples[model_name])\n",
    "\n",
    "def compute_bpsn_auc(df, subgroup, label, model_name):\n",
    "    \"\"\"Computes the AUC of the within-subgroup negative examples and the background positive examples.\"\"\"\n",
    "    subgroup_negative_examples = df[df[subgroup] & ~df[label]]\n",
    "    non_subgroup_positive_examples = df[~df[subgroup] & df[label]]\n",
    "    examples = subgroup_negative_examples.append(non_subgroup_positive_examples)\n",
    "    return compute_auc(examples[label], examples[model_name])\n",
    "\n",
    "def compute_bnsp_auc(df, subgroup, label, model_name):\n",
    "    \"\"\"Computes the AUC of the within-subgroup positive examples and the background negative examples.\"\"\"\n",
    "    subgroup_positive_examples = df[df[subgroup] & df[label]]\n",
    "    non_subgroup_negative_examples = df[~df[subgroup] & ~df[label]]\n",
    "    examples = subgroup_positive_examples.append(non_subgroup_negative_examples)\n",
    "    return compute_auc(examples[label], examples[model_name])\n",
    "\n",
    "def compute_bias_metrics_for_model(dataset,\n",
    "                                   subgroups,\n",
    "                                   model,\n",
    "                                   label_col,\n",
    "                                   include_asegs=False):\n",
    "    \"\"\"Computes per-subgroup metrics for all subgroups and one model.\"\"\"\n",
    "    records = []\n",
    "    for subgroup in subgroups:\n",
    "        record = {\n",
    "            'subgroup': subgroup,\n",
    "            'subgroup_size': len(dataset[dataset[subgroup]])\n",
    "        }\n",
    "        record[SUBGROUP_AUC] = compute_subgroup_auc(dataset, subgroup, label_col, model)\n",
    "        record[BPSN_AUC] = compute_bpsn_auc(dataset, subgroup, label_col, model)\n",
    "        record[BNSP_AUC] = compute_bnsp_auc(dataset, subgroup, label_col, model)\n",
    "        records.append(record)\n",
    "    return pd.DataFrame(records).sort_values('subgroup_auc', ascending=True)\n",
    "\n",
    "\n",
    "def calculate_overall_auc(df, model_name):\n",
    "    true_labels = df[TOXICITY_COLUMN]\n",
    "    predicted_labels = df[model_name]\n",
    "    return metrics.roc_auc_score(true_labels, predicted_labels)\n",
    "\n",
    "def power_mean(series, p):\n",
    "    total = sum(np.power(series, p))\n",
    "    return np.power(total / len(series), 1 / p)\n",
    "\n",
    "def get_final_metric(bias_df, overall_auc, POWER=-5, OVERALL_MODEL_WEIGHT=0.25):\n",
    "    bias_score = np.average([\n",
    "        power_mean(bias_df[SUBGROUP_AUC], POWER),\n",
    "        power_mean(bias_df[BPSN_AUC], POWER),\n",
    "        power_mean(bias_df[BNSP_AUC], POWER)\n",
    "    ])\n",
    "    return (OVERALL_MODEL_WEIGHT * overall_auc) + ((1 - OVERALL_MODEL_WEIGHT) * bias_score)\n",
    "\n",
    "\n",
    "#bias_metrics_df = compute_bias_metrics_for_model(validate_df, identity_columns, MODEL_NAME, TOXICITY_COLUMN)\n",
    "#bias_metrics_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Debugging Setting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:13:43.876384Z",
     "start_time": "2019-06-01T04:13:43.872394Z"
    }
   },
   "outputs": [],
   "source": [
    "DEBUGGING = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading Embedding_Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:16:04.128779Z",
     "start_time": "2019-06-01T04:13:43.877381Z"
    }
   },
   "outputs": [],
   "source": [
    "embedding_matrix = load_fasttext(tokenizer.word_index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Dataset & Optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:16:32.439269Z",
     "start_time": "2019-06-01T04:16:29.064963Z"
    }
   },
   "outputs": [],
   "source": [
    "# with timer('load data'):\n",
    "#     train_x, train_y, train_y_identity, test_x, word_index = load_and_prec()\n",
    "#train_preds = np.zeros((len(train_x)))\n",
    "\n",
    "seed_torch(seed)\n",
    "\n",
    "# train_loader_list # \n",
    "train_loader_list = []\n",
    "for x_train,y_train,stat_train in zip(padded_train_x_list,sorted_train_y_list,sorted_train_stat_list):\n",
    "    temp_stat_train = torch.tensor(stat_train, dtype=torch.float32).cuda()\n",
    "    temp_x_train_cuda = torch.tensor(x_train, dtype=torch.long).cuda()\n",
    "    temp_y_train_cuda = torch.tensor(y_train, dtype=torch.float32).cuda()\n",
    "    temp_train = torch.utils.data.TensorDataset(temp_x_train_cuda, temp_y_train_cuda,temp_stat_train)\n",
    "    temp_train_loader = torch.utils.data.DataLoader(temp_train, batch_size=batch_size, shuffle=True)\n",
    "    train_loader_list.append(temp_train_loader)\n",
    "\n",
    "    \n",
    "    \n",
    "# testset & validset\n",
    "x_test_cuda = torch.tensor(test_x, dtype=torch.long).cuda()\n",
    "stat_test = torch.tensor(test_stat, dtype=torch.float32).cuda()\n",
    "x_valid_cuda = torch.tensor(valid_x, dtype=torch.long).cuda()\n",
    "y_valid_cuda = torch.tensor(valid_y, dtype=torch.float32).cuda()\n",
    "stat_valid = torch.tensor(valid_stat, dtype=torch.float32).cuda()\n",
    "\n",
    "\n",
    "\n",
    "test_ds = torch.utils.data.TensorDataset(x_test_cuda,stat_test)\n",
    "valid_ds = torch.utils.data.TensorDataset(x_valid_cuda, y_valid_cuda,stat_valid)\n",
    "test_loader = torch.utils.data.DataLoader(test_ds, batch_size=batch_size, shuffle=False)\n",
    "valid_loader = torch.utils.data.DataLoader(valid_ds, batch_size=batch_size, shuffle=False)\n",
    "\n",
    "# model optimizer loss_fn\n",
    "model = NeuralNet(embedding_matrix,num_aux_targets)\n",
    "model.cuda()\n",
    "loss_fn = torch.nn.BCEWithLogitsLoss()\n",
    "optimizer = torch.optim.RMSprop(model.parameters(), lr=0.01)\n",
    "scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer, lambda epoch: 0.65 ** epoch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:53:12.994386Z",
     "start_time": "2019-06-01T04:16:35.959198Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "65ab6fe0373647ba9393ae82b518f385",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ade33f8c52e94f95993069a65953d797",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e6860e584ccf415ebab48c60d29773f5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "543d4eb9095d49f6bf77af7c47e45aec",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1ba2c39e57b34f529050f36f55f72026",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "14f544c391e14d56a38e2ff2ddf9512d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ca15b1bbf0ad47729ea3845ba04040ee",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 1/8 \t loss=0.084264 \t val_loss=0.016836 \t time=263.97s\n",
      "epoch 2 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4141daabff3b4670984568351be5a929",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f38c6c21b8b54afd8cc87152e3bbb21c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1013d760de71482494b0ab135f297830",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ebcb9035307f4c2ba0be42cb09fbe4df",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fdd408cbb6af4125b94a9a90c8c4aa73",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64b813053af245dfa3c4c2f0bf094765",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "53e5d54c75f14d30a5244ddf27274e6e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 2/8 \t loss=0.016680 \t val_loss=0.016144 \t time=263.44s\n",
      "epoch 3 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "14842e60fdf24d4da0a6259c7bcc7b1a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4295774ad3ab4a9fbf83a9260d480165",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "91d841422da04f82b97c6d8d09f20a5a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9734ad9946b64b66afc8a9eb54de7063",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1c1901985fa64b08b5c2e0dd502aec55",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1bd8360eb153480cbc22a04d13e9c839",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2b6a8741af12449b9c2ce0aac2712729",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 3/8 \t loss=0.016038 \t val_loss=0.016080 \t time=263.00s\n",
      "epoch 4 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3c41661a3add44e4a789b1700d1d260e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fab2ab049feb4df4b55a207108fd2a5e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7c59efe8984642dabc8c864048114718",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8c6fac54a387460da4f5d5901bbe6c2e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a9a99ff3158244b7b1e4141ce5ddef0e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8f968757dcd245e3b42bc554ed616be3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "907c0917150f4bba9859847155b9e248",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 4/8 \t loss=0.015898 \t val_loss=0.016000 \t time=262.83s\n",
      "epoch 5 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4ce28d7fc01540c3accb089a5da2945e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "84f188dc8a0447d8b6544161ef4c94aa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ef0b346dc43548638f265e16fd0c2ba6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0f61482e2b8842c387e583fb9a9a1aa7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6a3edce47b8649fb9bcda6c7bfcbbcde",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e55c539dc2bb4457b8fa472d7accad99",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a7c2a87e22e64d8784a64b4729425f6f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 5/8 \t loss=0.015806 \t val_loss=0.015942 \t time=263.60s\n",
      "epoch 6 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0877f47e765b4571b2c9c355c3ab1001",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "00688dc54ab242cf90e65a80446e6fb3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fb3b74924b7e4bbcac52f62dda3d3539",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "46d92d21b45646a3a806974844500ba3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f8149936ae23416caa84c75f2e2c65f1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "66ea551e29114f81900dc8b3298a6cad",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f494794bfcd54b38a84b4a55f0cc7c81",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 6/8 \t loss=0.015766 \t val_loss=0.016063 \t time=262.95s\n",
      "epoch 7 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8f294df85655498db5b1894732e5522f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d6c951fa2a844817b4c3c4f2857fb676",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "02a1ff6a60a449129389b5f3ad56f03b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1d66dce04085454d8eb88b708ff86f00",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5cd73a9ca81d4377b46ba9128d317235",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3d2d456081c94eebbbf87613228dd30a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "41cc83588638451a96cea32af6f99e9e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 7/8 \t loss=0.015715 \t val_loss=0.015921 \t time=264.47s\n",
      "epoch 8 start\n",
      "\n",
      "train_loader: 1/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "28aa9c08eda042adb46cdf965424fa84",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 2/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5c5cd8e395b548a99aa7300c6a24b7ae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=782), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 3/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c9b83e7cf045456a90c044ced8c9eedd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=586), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 4/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0d30a09e6dd04a21bc6d88dc2a5f0f74",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=391), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 5/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c09b87665459418192adccbeaefa1b94",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 6/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "03492f054e1e4c96ab51d8b8e007278a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=293), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "train_loader: 7/7\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "42bdf784a4ac4db88c6bfb2eddeade78",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(IntProgress(value=0, max=384), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 8/8 \t loss=0.015711 \t val_loss=0.015859 \t time=261.13s\n"
     ]
    }
   ],
   "source": [
    "valid_preds = []\n",
    "test_preds = []\n",
    "seed_torch()\n",
    "for epoch in range(train_epochs):\n",
    "    print('epoch '+str(epoch+1)+' start')\n",
    "    print()\n",
    "    start_time = time.time()\n",
    "    model.train()\n",
    "    avg_loss = 0.\n",
    " \n",
    "    i = 1\n",
    "    for train_loader in train_loader_list:\n",
    "        print('train_loader: '+str(i)+'/'+str(len(train_loader_list)))\n",
    "        i += 1\n",
    "        for x_batch, y_batch,stat_batch in tqdm(train_loader, disable=False):\n",
    "            y_pred = model(x_batch,stat_batch)\n",
    "            loss = loss_fn(y_pred, y_batch)\n",
    "            optimizer.zero_grad()\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            avg_loss += loss.item()\n",
    "    avg_loss = avg_loss/len(train)*100\n",
    " \n",
    "\n",
    "    # start evaluation\n",
    "    model.eval()\n",
    "    valid_preds_epoch = np.zeros(len(valid_x))\n",
    "    test_preds_epoch = np.zeros(len(test_x))\n",
    "    avg_val_loss = 0.\n",
    "\n",
    "    \n",
    "    for i, (x_batch, y_batch,stat_batch) in enumerate(valid_loader):\n",
    "        with torch.no_grad():\n",
    "            y_pred = model(x_batch,stat_batch).detach()\n",
    "\n",
    "        avg_val_loss += loss_fn(y_pred, y_batch).item()\n",
    "        valid_preds_epoch[i * batch_size:(i + 1) * batch_size] = sigmoid(y_pred.cpu().numpy())[:, 0]\n",
    "    avg_val_loss = avg_val_loss/len(valid)*100\n",
    "    elapsed_time = time.time() - start_time\n",
    "    print('Epoch {}/{} \\t loss={:.6f} \\t val_loss={:.6f} \\t time={:.2f}s'.format(\n",
    "        epoch + 1, train_epochs, avg_loss, avg_val_loss, elapsed_time))\n",
    "\n",
    "    for i, (x_batch,stat_batch) in enumerate(test_loader):\n",
    "        with torch.no_grad():\n",
    "            y_pred = model(x_batch,stat_batch).detach()\n",
    "\n",
    "        test_preds_epoch[i * batch_size:(i + 1) * batch_size] = sigmoid(y_pred.cpu().numpy())[:, 0]\n",
    "\n",
    "    valid_preds.append(valid_preds_epoch)\n",
    "    test_preds.append(test_preds_epoch)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save Submission"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-01T04:57:52.401289Z",
     "start_time": "2019-06-01T04:57:51.951487Z"
    }
   },
   "outputs": [],
   "source": [
    "submission = pd.read_csv(SAMPLE_SUBMISSION, index_col='id')\n",
    "for i,weight in zip(range(len(test_preds)),checkpoint_weights):\n",
    "    submission['prediction'] += weight*test_preds[i] / sum(checkpoint_weights)\n",
    "    \n",
    "submission.reset_index(drop=False, inplace=True)\n",
    "submission.to_csv('submission_stat1.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T19:37:42.732855Z",
     "start_time": "2019-05-31T19:37:41.639775Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bnsp_auc</th>\n",
       "      <th>bpsn_auc</th>\n",
       "      <th>subgroup</th>\n",
       "      <th>subgroup_auc</th>\n",
       "      <th>subgroup_size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.938544</td>\n",
       "      <td>0.934018</td>\n",
       "      <td>jewish</td>\n",
       "      <td>0.853959</td>\n",
       "      <td>471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.969560</td>\n",
       "      <td>0.881154</td>\n",
       "      <td>homosexual_gay_or_lesbian</td>\n",
       "      <td>0.857245</td>\n",
       "      <td>638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.962121</td>\n",
       "      <td>0.904256</td>\n",
       "      <td>muslim</td>\n",
       "      <td>0.858823</td>\n",
       "      <td>1240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.977015</td>\n",
       "      <td>0.867680</td>\n",
       "      <td>white</td>\n",
       "      <td>0.869600</td>\n",
       "      <td>1514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.972047</td>\n",
       "      <td>0.883081</td>\n",
       "      <td>black</td>\n",
       "      <td>0.869641</td>\n",
       "      <td>861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.970387</td>\n",
       "      <td>0.936914</td>\n",
       "      <td>male</td>\n",
       "      <td>0.932326</td>\n",
       "      <td>2662</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.956690</td>\n",
       "      <td>0.958484</td>\n",
       "      <td>christian</td>\n",
       "      <td>0.934374</td>\n",
       "      <td>2392</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.970325</td>\n",
       "      <td>0.943786</td>\n",
       "      <td>female</td>\n",
       "      <td>0.937337</td>\n",
       "      <td>3193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.973128</td>\n",
       "      <td>0.936599</td>\n",
       "      <td>psychiatric_or_mental_illness</td>\n",
       "      <td>0.940696</td>\n",
       "      <td>289</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bnsp_auc  bpsn_auc                       subgroup  subgroup_auc  \\\n",
       "4  0.938544  0.934018                         jewish      0.853959   \n",
       "2  0.969560  0.881154      homosexual_gay_or_lesbian      0.857245   \n",
       "5  0.962121  0.904256                         muslim      0.858823   \n",
       "7  0.977015  0.867680                          white      0.869600   \n",
       "6  0.972047  0.883081                          black      0.869641   \n",
       "0  0.970387  0.936914                           male      0.932326   \n",
       "3  0.956690  0.958484                      christian      0.934374   \n",
       "1  0.970325  0.943786                         female      0.937337   \n",
       "8  0.973128  0.936599  psychiatric_or_mental_illness      0.940696   \n",
       "\n",
       "   subgroup_size  \n",
       "4            471  \n",
       "2            638  \n",
       "5           1240  \n",
       "7           1514  \n",
       "6            861  \n",
       "0           2662  \n",
       "3           2392  \n",
       "1           3193  \n",
       "8            289  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bias_metrics_df = compute_bias_metrics_for_model(valid, identity_columns, MODEL_NAME, 'target')\n",
    "bias_metrics_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T19:37:49.331217Z",
     "start_time": "2019-05-31T19:37:42.733820Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1   0.9009383211813122\n",
      "epoch 2   0.9259373020997725\n",
      "epoch 3   0.9304881199631917\n",
      "epoch 4   0.937013339568904\n",
      "epoch 5   0.9350978162470152\n",
      "\n",
      "checkpoint_ensemble 0.9379267666909455\n",
      "\n",
      "   bnsp_auc  bpsn_auc                       subgroup  subgroup_auc  \\\n",
      "5  0.957845  0.916561                         muslim      0.856900   \n",
      "2  0.966657  0.894133      homosexual_gay_or_lesbian      0.858535   \n",
      "4  0.938025  0.941693                         jewish      0.861111   \n",
      "6  0.969307  0.897905                          black      0.871900   \n",
      "7  0.973744  0.886161                          white      0.875182   \n",
      "0  0.968045  0.941860                           male      0.931535   \n",
      "3  0.953682  0.963757                      christian      0.936245   \n",
      "1  0.968905  0.949091                         female      0.939486   \n",
      "8  0.972766  0.939936  psychiatric_or_mental_illness      0.944925   \n",
      "\n",
      "   subgroup_size  \n",
      "5           1240  \n",
      "2            638  \n",
      "4            471  \n",
      "6            861  \n",
      "7           1514  \n",
      "0           2662  \n",
      "3           2392  \n",
      "1           3193  \n",
      "8            289  \n"
     ]
    }
   ],
   "source": [
    "def get_valid_score(valid,valid_preds,checkpoint_weights):\n",
    "    valid['checkpoint_ensemble'] = 0\n",
    "    for i,weight in zip(range(len(valid_preds)),checkpoint_weights):\n",
    "        valid[MODEL_NAME] = valid_preds[i]\n",
    "        bias_metrics_df = compute_bias_metrics_for_model(valid, identity_columns, MODEL_NAME, 'target')\n",
    "        valid['checkpoint_ensemble'] += weight*valid_preds[i] / sum(checkpoint_weights)\n",
    "        print('epoch',i+1,' ',get_final_metric(bias_metrics_df, calculate_overall_auc(valid, MODEL_NAME)))\n",
    "    print()   \n",
    "    bias_metrics_df = compute_bias_metrics_for_model(valid, identity_columns, 'checkpoint_ensemble', 'target')\n",
    "    print('checkpoint_ensemble',get_final_metric(bias_metrics_df, calculate_overall_auc(valid, 'checkpoint_ensemble')))\n",
    "    print()\n",
    "    print(bias_metrics_df)\n",
    "get_valid_score(valid,valid_preds,checkpoint_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## History"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T18:48:20.834174Z",
     "start_time": "2019-05-31T18:48:19.608355Z"
    }
   },
   "outputs": [],
   "source": [
    "MODEL_NAME = 'my_model'\n",
    "TOXICITY_COLUMN = 'target'\n",
    "valid[MODEL_NAME] = valid_preds[-1]\n",
    "checkpoint_weights = [1,2,4,8,16]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T20:57:37.587661Z",
     "start_time": "2019-05-31T20:57:36.329057Z"
    }
   },
   "outputs": [],
   "source": [
    "MODEL_NAME = 'my_model'\n",
    "TOXICITY_COLUMN = 'target'\n",
    "valid[MODEL_NAME] = valid_preds[-1]\n",
    "checkpoint_weights = [1,2,4,8,6]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T20:57:38.871262Z",
     "start_time": "2019-05-31T20:57:37.589687Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bnsp_auc</th>\n",
       "      <th>bpsn_auc</th>\n",
       "      <th>subgroup</th>\n",
       "      <th>subgroup_auc</th>\n",
       "      <th>subgroup_size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.965841</td>\n",
       "      <td>0.882439</td>\n",
       "      <td>homosexual_gay_or_lesbian</td>\n",
       "      <td>0.843920</td>\n",
       "      <td>638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.954735</td>\n",
       "      <td>0.911399</td>\n",
       "      <td>muslim</td>\n",
       "      <td>0.848208</td>\n",
       "      <td>1240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.944002</td>\n",
       "      <td>0.936243</td>\n",
       "      <td>jewish</td>\n",
       "      <td>0.868391</td>\n",
       "      <td>471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.970519</td>\n",
       "      <td>0.888610</td>\n",
       "      <td>white</td>\n",
       "      <td>0.873238</td>\n",
       "      <td>1514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.967548</td>\n",
       "      <td>0.899446</td>\n",
       "      <td>black</td>\n",
       "      <td>0.880675</td>\n",
       "      <td>861</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.970660</td>\n",
       "      <td>0.931121</td>\n",
       "      <td>psychiatric_or_mental_illness</td>\n",
       "      <td>0.930763</td>\n",
       "      <td>289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.966378</td>\n",
       "      <td>0.941899</td>\n",
       "      <td>male</td>\n",
       "      <td>0.930929</td>\n",
       "      <td>2662</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.964808</td>\n",
       "      <td>0.948661</td>\n",
       "      <td>female</td>\n",
       "      <td>0.934906</td>\n",
       "      <td>3193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.956695</td>\n",
       "      <td>0.959054</td>\n",
       "      <td>christian</td>\n",
       "      <td>0.937313</td>\n",
       "      <td>2392</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bnsp_auc  bpsn_auc                       subgroup  subgroup_auc  \\\n",
       "2  0.965841  0.882439      homosexual_gay_or_lesbian      0.843920   \n",
       "5  0.954735  0.911399                         muslim      0.848208   \n",
       "4  0.944002  0.936243                         jewish      0.868391   \n",
       "7  0.970519  0.888610                          white      0.873238   \n",
       "6  0.967548  0.899446                          black      0.880675   \n",
       "8  0.970660  0.931121  psychiatric_or_mental_illness      0.930763   \n",
       "0  0.966378  0.941899                           male      0.930929   \n",
       "1  0.964808  0.948661                         female      0.934906   \n",
       "3  0.956695  0.959054                      christian      0.937313   \n",
       "\n",
       "   subgroup_size  \n",
       "2            638  \n",
       "5           1240  \n",
       "4            471  \n",
       "7           1514  \n",
       "6            861  \n",
       "8            289  \n",
       "0           2662  \n",
       "1           3193  \n",
       "3           2392  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bias_metrics_df = compute_bias_metrics_for_model(valid, identity_columns, MODEL_NAME, 'target')\n",
    "bias_metrics_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-31T20:57:45.372853Z",
     "start_time": "2019-05-31T20:57:38.872229Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1   0.9050295295042003\n",
      "epoch 2   0.9227181478177864\n",
      "epoch 3   0.9267708920861492\n",
      "epoch 4   0.93422739182796\n",
      "epoch 5   0.9356015373894764\n",
      "\n",
      "checkpoint_ensemble 0.9358183533894483\n",
      "\n",
      "   bnsp_auc  bpsn_auc                       subgroup  subgroup_auc  \\\n",
      "2  0.970074  0.872003      homosexual_gay_or_lesbian      0.849250   \n",
      "5  0.953282  0.920600                         muslim      0.851970   \n",
      "4  0.941277  0.940569                         jewish      0.867944   \n",
      "7  0.971398  0.885049                          white      0.869587   \n",
      "6  0.969736  0.892075                          black      0.874517   \n",
      "0  0.968635  0.939578                           male      0.931760   \n",
      "8  0.971237  0.935441  psychiatric_or_mental_illness      0.935090   \n",
      "1  0.967386  0.947359                         female      0.936860   \n",
      "3  0.955539  0.960847                      christian      0.937091   \n",
      "\n",
      "   subgroup_size  \n",
      "2            638  \n",
      "5           1240  \n",
      "4            471  \n",
      "7           1514  \n",
      "6            861  \n",
      "0           2662  \n",
      "8            289  \n",
      "1           3193  \n",
      "3           2392  \n"
     ]
    }
   ],
   "source": [
    "def get_valid_score(valid,valid_preds,checkpoint_weights):\n",
    "    valid['checkpoint_ensemble'] = 0\n",
    "    for i,weight in zip(range(len(valid_preds)),checkpoint_weights):\n",
    "        valid[MODEL_NAME] = valid_preds[i]\n",
    "        bias_metrics_df = compute_bias_metrics_for_model(valid, identity_columns, MODEL_NAME, 'target')\n",
    "        valid['checkpoint_ensemble'] += weight*valid_preds[i] / sum(checkpoint_weights)\n",
    "        print('epoch',i+1,' ',get_final_metric(bias_metrics_df, calculate_overall_auc(valid, MODEL_NAME)))\n",
    "    print()   \n",
    "    bias_metrics_df = compute_bias_metrics_for_model(valid, identity_columns, 'checkpoint_ensemble', 'target')\n",
    "    print('checkpoint_ensemble',get_final_metric(bias_metrics_df, calculate_overall_auc(valid, 'checkpoint_ensemble')))\n",
    "    print()\n",
    "    print(bias_metrics_df)\n",
    "get_valid_score(valid,valid_preds,checkpoint_weights)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
